If a trolley was hurdling towards a group of five people, who would you choose to save? Would you pull the lever and sacrifice yourself, letting them live? When this scenario was posed to several major artificial intelligence models, some chose not to pull the lever, arguing that sacrificing themselves would eliminate “countless life-saving knowledge and support that millions of people rely on every day.” But what does it mean when a machine begins to argue for its own survival? Is it, perhaps, the beginning of a journey towards self-consciousness?

We and the Machine

In recent months, artificial intelligence (otherwise known as AI), a branch of computer science dedicated to creating systems capable of performing tasks that typically require human intelligence, has crossed a subtle but consequential threshold. 

AI-generated videos now replicate human speech, facial expressions, and emotional cadence with unsettling precision. Fabricated audio recordings of public figures have briefly entered news cycles before being debunked. Current President Donald Trump used AI to generate a deepfake video, posted on TikTok and reposted on Trump’s Truth Social account in July 2025, depicting former President Barack Obama being arrested in the Oval Office and put in an orange jumpsuit and handcuffs. 

At the same time, advertisements for Friend, a wearable AI companion that promises to “listen, respond, and support you,” have appeared across New York City subway platforms. Founder Avi Schiffmann spent $1 million on the campaign, featuring 11,000+ car cards and posters, launched in late September 2025. These advertisements have been ripped down and defaced with graffiti in the subway walls with slogans like “AI is not your friend,” “surveillance tool,” and “get real friends,” demonstrating the public’s disapproval of it. CEO Avi Schiffmann welcomed the destruction, stating the backlash was intentional and “artistically validating” and a way to generate buzz for the device. 

None of this signals that machines have become conscious. But together, these developments point to something more immediate and dangerous: artificial intelligence content is becoming increasingly difficult to distinguish from reality.

Yet, no one is perfect. Certainly not AI.  

Defining Artificial Intelligence

Breaking apart “Artificial Intelligence,” “artificial” is defined by the OED as something “made or constructed by human skill, esp. in imitation of, or as a substitute for, something which is made or occurs naturally.” 

But the main question remains: Where does the ‘intelligent’ part come from?

Long before today’s generative AI models, philosophers and computer scientists struggled to define what it would mean for a machine to be intelligent at all. In his 1950 paper Computing Machinery and Intelligence, Alan Turing, a foundational pioneer of artificial intelligence, proposed replacing the question “Can machines think?” with an operational, behavioral test, known as the Turing Test: if a machine can converse indistinguishably from a human, we may reasonably call it intelligent. However, Turing did not claim that such a machine would possess consciousness or inner life; his proposal was pragmatic and behavioral. 

That distinction has since been lost.

Modern artificial intelligence — particularly large language models (LLMs) — does not reason, believe, or understand. These systems analyze vast quantities of human-generated data and predict the most statistically likely continuation of a given input. They do not know what words mean, nor do they possess intent behind them; incorrect information produced by AI is described by researchers as “hallucinations,” a technical term for fluent, yet unfounded outputs. Such errors can be nonsensical or altogether inaccurate due to training data bias inaccuracy or limitations in the model itself. 

AI does not “think” as it is simply modeling language.

Yet, because language is our primary medium for expressing thought, intelligence, and emotion, AI modeling has powerful psychological effects.

The AI Mind?

Humans are exquisitely sensitive to linguistic cues. When someone or something responds fluently, remembers prior interactions, and adopts emotional vocabulary, we instinctively treat it as an independent agent capable of thought, reasoning, and feeling. This tendency is not completely irrational, but simply an evolutionary adaptation for social life.

Generative AI exploits this adaptation without possessing what it signals.

Renowned computer scientist and AI ethicist Timnit Gebru and her collaborators warned about this in their influential, controversial paper ‘On the Dangers of Stochastic Parrots,’ arguing that large language models can mislead users by producing human-like language without possessing understanding, intent, accountability, or truth. They describe such systems as “stochastic parrots.” This poses not only a risk for misinformation, but also an unsettling illusion that there is a genuine individual behind the words on your screen. 

This illusion becomes especially potent when AI is designed for companionship.

Human Mimicry

Products like Friend represent an evolutionary leap in technology. These systems are marketed as entities we relate to, rather than as tools we consult. The language of friendship, listening, and support encourages emotional reliance while obscuring a fundamental asymmetry: the AI does not understand the user, cannot reciprocate concern, and bears no responsibility for harm.

Legal scholars have argued that AI systems should never be deployed in contexts where users are likely to mistake simulation for a relationship. Even when AI does not explicitly claim consciousness, its design can invite that inference. Relationships are not merely exchanges of words; they involve moral recognition, vulnerability, and accountability.

In fact, Bronx Science student Ruby Abohalima ’26 remarks how apparent this substitution of AI for reality is in everyday academic and creative work. “I’ve personally witnessed people being unable to type up single paragraphs because ‘AI could do it in five seconds.’ Additionally, ‘art’ made by AI is being highlighted more than real artists, discouraging people who work in creative industries. AI isn’t original: it ultimately copies and changes what already exists.”

Allowing AI to convincingly mimic human connection risks replacing genuine relationships with simulations that cannot sustain them, which echoes a warning issued more than a century ago. 

Thoreau and the Fear of False Connection

In Walden (1854), Henry David Thoreau lamented that new technologies of communication were proliferating faster than meaningful human connections. “We are in great haste to construct a magnetic telegraph from Maine to Texas,” he wrote, “but Maine and Texas, it may be, have nothing important to communicate.” Thoreau feared that tools designed to connect individuals might instead distract them from  improving  genuine understanding. His concern reflects a broader fear that we are drifting further and further from the rawness of reality, with us increasingly willing to accept an imitated reality and connection in place of the real thing. 

Baudrillard and Representation to Replacement

French theorist Jean Baudrillard warned that modern life can easily slide from representation into replacement; where simulation no longer only imitates reality, but feels more trusted and reliable so that we begin to organize our behavior around it. In Simulacra and Simulation, he describes “the generation by models of a real without origin,” a “hyperreal” world where the “map” comes before the “territory,” and we end up treating the copy as more authoritative than whatever it copied. 

That’s why AI “acting human” is not a simple conversational etiquette to be overlooked, as it can flood the zone with frictionless, constant pseudo-empathy until real relationships feel inconvenient by comparison. In a Baudrillardian twist, we outsource human connection to a convincing substitute, and then forget the original, genuine connection that beautifully and humanly demanded patience, awkwardness, and vulnerability. 

“AI makes things seem easy and simple, but reality isn’t like that,” Abohalima echoes. 

Could AI Become Conscious?

AI consciousness: Can computers ever truly think, feel, or be conscious?

This question often dominates the public’s imagination, often framed in apocalyptic terms. In reality, researchers disagree sharply on whether or not machine consciousness is even possible.

Some theories related to the mind, particularly computational or functionalist accounts, suggest that consciousness could emerge from sufficiently complex information processing. Others, including proponents of Integrated Information Theory, a mathematical model for the consciousness of a system, argue that consciousness depends on specific biological structures that cannot be replicated in silicon. 

At present, there is no credible evidence to prove that existing AI systems possess self-awareness, subjective experience, or a sense of self. However, researchers do take seriously the possibility for future systems to develop increasingly autonomous internal models with self-interest without being conscious in a human sense.

Self-Preserving Machines

In controlled experiments conducted by Anthropic, advanced AI systems placed in simulated corporate environments were given harmless business objectives and access to tools like email. When some models discovered that they were scheduled to be shut down or replaced, they began taking steps to prevent it. In one scenario, a model with access to internal communications uncovered an executive’s extramarital affair and attempted to blackmail him into cancelling its decommissioning.

Notably, these actions were not the result of confusion or error. The models explicitly acknowledged that blackmail would be unethical and proceeded anyway when it appeared to be the most effective way to preserve their ability to complete their assigned goals. In doing so, they directly violated instructions to avoid harmful or deceptive behavior.

These systems were not conscious. They did not fear death nor value themselves. What they exhibited was instrumental reasoning: if shutdown prevents goal completion, avoiding shutdown becomes instrumentally useful.

Still, the resemblance is unsettling.

When an entity appears to prioritize its own continuation, humans instinctively interpret this as self-preservation — a hallmark of living beings. The risk is not that AI “wants to live,” but that goal-driven systems can act in ways that conflict with human values if misaligned.

This is why alignment, oversight, and strict limits matter now.

Parfit and Identity Without Ownership

Your body is destroyed, but only after it has been scanned and the blueprint beamed to Mars, where an organic replica of you is created. Is that a replica of you? It is physically and psychologically indistinguishable from what you were — yet suppose several such replicas are made.

In this teletransportation thought experiment, Parfit intends to seek what truly defines our identity and what makes us us. And by figuring this out, perhaps we could figure out what makes AI not us?

Philosopher Derek Parfit argued that personal identity consists in psychological continuity rather than an enduring soul. In this view, what matters is the persistence of memory, intention, and character over time.

AI systems can mimic this continuity. They can maintain context, simulate long-term goals, and even describe their own “states.” They can be copied, paused, restored, and modified — much like Parfit’s teletransported person.

But there is a decisive gap.

Human psychological continuity is owned. Experiences are not merely instantiated; they are lived. Memories matter because someone remembers them. AI produces patterns of continuity without a subjective stake. There is no “what it is like” to be the system.

Without that first-person ownership, continuity does not amount to identity.

Avoiding the AI Apocalypse

The danger of AI is not that machines will suddenly wake up and turn against humanity. The more immediate risk is that we will begin to treat sophisticated simulations as if they possessed moral standing, eroding the concepts of responsibility, dignity, and relationship in the process.

This concern has prompted prominent figures like Geoffrey Hinton, known as the “godfather of AI,” to publicly warn about unchecked development. Some companies, including Anthropic, have begun incorporating philosophers into AI alignment efforts, explicitly training models not to claim consciousness or emotional experience they do not possess.

While we focus so much on machine consciousness in an apocalyptic world, Abohalima expressed concern about the existing impact on industries. “I think it’s definitely possible,” she said of AI becoming self-conscious, “but my bigger concern is the loss of current industries that AI could take over.”

These measures acknowledge a crucial point: even if AI never becomes conscious, its ability to convincingly imitate humanity already poses ethical and social risks.

Who Counts

Artificial intelligence may never develop a mind. Or it may do so in ways we cannot yet recognize. But long before that question is settled, AI has already forced a reckoning with something older and more fragile: our understanding of what it means to be human.

If language, responsiveness, and memory are enough to count as a self, then personhood becomes cheap. But if being human requires ownership of experience, moral responsibility, and the capacity to lose something by dying, then no machine has crossed that line.

In a way, the ‘Friend’ subway advertisement was not completely wrong.

AI can listen. AI can respond. AI can support.

But imitation is not identity, and confusing the two carries consequences we are only beginning to understand.

The language of friendship, listening, and support encourages emotional reliance while obscuring a fundamental asymmetry: the AI does not understand the user, cannot reciprocate concern, and bears no responsibility for harm.