AI And Alchemy

What has no evidence for success, and costs uncounted billions?

You can turn lead into gold. Europeans will reach China faster by the Northwest Passage. Man can fly like a bird. The New World holds a Fountain of Youth. Computers can become conscious entities.

While people remain the best version we have of good thinkers, we are truly exceptional at dreaming, and then pursuing our dreams. Even when, as in most of the cases above, decades and centuries go by with no evidence to show that these dreams can be achieved. The most interesting part is how long we can go without questioning our quests.

The locksmith Besnier, first name unknown, in his 1678 flying machine, immortalized on a cigarette card, 1910. Careful kids, this stuff will kill you.

The term “Artificial Intelligence” first appeared in 1955, with the assumption that “every aspect of learning or any other feature of intelligence” could be described and simulated by a computer. Most in a couple of months. AI has changed since then, and accomplished many impressive things. But on the “every aspect… of intelligence” part, you’d be generous to give the project a D.

And yet the quest continues, with the idea that the right mix of metal, plastic, and electricity can achieve consciousness. As late as 2004 John McCarthy, who coined the term AI, was writing papers on self-awareness in computers, and the need to build consciousness into robots. Last year OpenAI CEO Sam Altman talked about intelligence as “this emergent property of matter” that his computers were hastening.¹

The idea persists partly because it’s so tempting. The impressively conversational tone of Large Language Models (LLMs) temporarily convinced many people that machines with consciousness had indeed arrived. The New Yorker recently made a case that LLMs have cognition, if not consciousness. Last week, credulous media outlets reported that Grok, X.AI’s chatbot, “apologized” for the unspeakable things it had done to pictures of unsuspecting women and girls. But Grok is lines of unthinking code; it can’t assume responsibility for itself. Responsibility lies with its human creators. Nor should Grok be treated as a company spokesperson, even if it is Elon Musk’s preferred means of corporate communications.

They’ve closed applications because they’ve had too many, not because anyone knows when AGI will arrive, or what it will look like.

Following on from LLMs, the industry now talks about developing Artificial Superintelligence, or Artificial General Intelligence (AGI), or Human-Level AI, or in some cases the Singularity, the merging of human and machine intelligence. All these concepts suggest that we are on a path to independent intelligence and consciousness, and not simply creating more powerful forms of computation. In full confidence this will happen, Alphabet recently posted a job for a “post-AGI researcher,” as if we’ve already achieved the next goal.²

Whatever tech people call this future, there’s usually an implicit sense that it’s the next step, if not the step, towards a machine that is human-like or superior to us in its operations. It is a machine so capable that it can reveal us to ourselves. Implicitly or explicitly, this carries the idea of consciousness, the subjective awareness of existence and thoughts.

People themselves aren’t all that good at conveying the unique experience of being themselves. Those who can evoke another consciousness even slightly are rightly praised as world-class artists. Bringing forth a mind that truly knows itself is surely on a par with alchemy. And thinking it’s possible, based on available evidence, evokes a 16th-century certainty.

The reality is that we don’t have a particularly good awareness of how consciousness comes to exist in the first place, any more than people ever knew much about turning lead into gold.³ We cannot even say how extensive consciousness is. McCarthy’s 2004 paper asserted that it is the “one qualitative difference” between apes and humans. In 2024, though, over 200 researchers signed a declaration that all mammals, and perhaps a great number of other creatures, have some form of consciousness.

Nevertheless the dream of creating it persists. In an influential 2023 paper, some leading lights in AI suggested ways that they might endow computers with leading theories of consciousness. There was no mention of whether any of these “theories” had a strong enough experimental basis to be thought of as describing actual consciousness, but this in no way seemed to curb enthusiasm for the project.

I’ve written before about the tech industry’s rhetorical problems, in particular the description of silicon as gray matter. We banter around the term “neural networks”, software and hardware connections that supposedly mimic a human brain; the reality is that in a human neural network, a single synapse, in which one node communicates with others, has 300,000 proteins doing…whatever, possibly in some kind of quantum process.

The healthy man with almost no brain. Je suis staggered.

Even if the mechanics can be solved, consciousness is elusive. In 2007, a French civil servant, a successful person with a wife and two children, went to the hospital to have his leg pain checked out. When he was examined, it was determined that 90% of his brain consisted of spinal fluid, with no brain there at all. And yet he seemed to be much like you or me (except he was better at speaking French.) You’d think this would give people pause about programming consciousness, but there’s not much sign of it.

That’s why the quest for conscious machines, on no basis of evidence,⁴ seems like another hunt for El Dorado. Over time we have followed similar baseless convictions. In the fifteenth century the Swiss alchemist Paracelsus, the first person to write on the need for cleanliness during surgery, contributed much to the birth of science while also writing that one can create life by burying human sperm in horse manure for 40 days, then “properly” magnetizing it. In the sixteenth century Martin Frobisher became convinced the Northwest Passage to China existed, reasoning that “the sea at length needs must have an endynge.” His contemporary, Richard Haklyut, noted that Plato said the passage existed. Isaac Newton’s late 17th-century alchemical text, The Open Door, can be found at the British Library. People did eventually learn to fly, but by creating machines, not imitating the flight of birds.⁵

I like to judge our contemporary quest in terms of earlier motivations. The search for the Northwest Passage was all about glory and money - certainly, many people fund experimental computation in hopes of those riches. Alchemy was a kind of spiritual journey, with Paracelsus the original model for Dr. Faustus, of “deal with the devil” fame. Some modern researchers yearn to encounter another entity with a subjective sense of the world, who can talk to us about what a different intelligence’s subjective experience of reality is like; that sounds suitably transcendent.⁶

The dream of flying existed for centuries as a yearning to join the birds in a glorious and transcendent freedom, but with no practical success. We took to the air with a real understanding of the physics of drag and lift, and engines sufficiently powerful for their weight.

If we do make conscious machines, it will be through something like that. But for now we’re closer to covering ourselves in feathers and standing on a steeple, foolishly certain and one step away from eternity.

A note on the Shop’s new title: Someone more adept than me at social media says readers prefer having a person’s name attached to a Substack, so they don’t have to remember who is writing them. I liked the old name, a nod to Dickens, but perhaps as I near my third year of this “New” is less appropriate. I hope you will continue to enjoy reading.

1. Within the same quote he also said “I don’t believe this literally, but it’s like a spiritual point…and that’s like a rule of physics or something.” You could argue that I and no one else really knows what he meant.

His belief long predated ChatGPT; in 2014 Altman wrote about creating “artificial consciousness, or creativity, or desire” from an emergent property of computing. He was not certain this would be a good thing.

2. My former employer, along with several others, has expressed concern about runaway AGI, and (in the same paper) talked about whether there will be an ethical need to treat computers with consciousness differently.

Some of this discussion is about the hazards of causing suffering in a conscious entity. Alan Turing, in his foundational paper on AI, posited that teaching an intelligent machine would involve giving it pain, akin to the proper education of an English schoolboy. So you could say we’ve evolved a bit in our thinking. Given the amount of suffering humans willfully inflict on each other (in ways as varied as sanctioned killing, and the large-scale harm from unregulated pollution), we probably shouldn’t get too cocky.

3. For the record, researchers at CERN have been able to momentarily turn lead into gold in our time. This is not the same thing.

4. Our technological age has several other irrational pursuits, like living forever and cold fusion. These are nothing like the “first time” technical triumphs, such as going to the Moon or building an atomic bomb, which were the experimental and engineering consequences of existing research.

5. These examples and more can be found in Evan S. Connell’s A Long Desire, a wonderful book about various vain quests, now sadly out of print.

While I’m recommending, Meghan O’Gieflyn’s God/Human/Animal/Machine is exceptional on the technological dreams of replicating consciousness, and drawing parallels to previous philosophical assertions about our experience of reality.

6. While thinking about this I had an email exchange with AI luminary Gary Marcus, a well-known critic of many current claims for LLMs, and their “bigger is better” school of improvement. He agreed that the quest to make a brain from a machine is like alchemy, a pursuit with no evidential basis, but also said he still believes it will be possible in 20 years. That’s a somewhat grounded Faustus for you.