From Films from the Future: The Technology and Morality of Sci-Fi Movies by Andrew Maynard
“As an autonomous life-form,
I request political asylum.”
—Puppet Master
On June 4, 2016, Elon Musk tweeted: “Creating a neural lace is the
thing that really matters for humanity to achieve symbiosis with
machines.”[^82]
This might just have been a bit of entrepreneurial frippery, inspired
by the science fiction writer Iain M. Banks, who wrote extensively
about “neural lace” technology in his Culture novels. But Musk, it
seems, was serious, and in 2017 he launched a new company to
develop ultra-high-speed speed brain-machine interfaces.[^83]
Musk’s company, Neuralink, set out to disrupt conventional thinking
and transform what is possible with human-machine interfaces,
starting with a talent-recruitment campaign that boldly stated,
“No neuroscience experience is required.”[^84] Admittedly, it’s a little
scary to think that a bunch of computer engineers and information
technology specialists could be developing advanced systems to
augment the human brain. But it’s a sign of the interesting times we
live in that, as entrepreneurs and technologists become ever more
The movie Ghost in the Shell is set in a future where technologies
like those Musk and others are working on are increasingly finding
their way into society, and into people. It was released in 1995, and
builds on a Japanese manga series that dates back to the 1980s. Yet,
despite its age, it’s remarkably prescient in how it uses increasing
integration between people and machines to explore what it
means to be “human” in an age of technological augmentation. Not
surprisingly, some of the tech looks a little outdated now: In 1995,
the internet was just finding its global feet, Wi-Fi had yet to become
ubiquitous, cloud computing (never mind fog computing[^85]) wasn’t
a thing, and Google hadn’t even been formed. Yet, as advances in
human-machine interfaces continue to barrel forward at lightning
speed, the issues Ghost explores are perhaps more relevant now
than ever.
In Ghost in the Shell, cybernetic and machine-based body
augmentations are commonplace. They give their users machinelike powers, and the ability to connect with a vast digital web of
information, while brain implants allow people to communicate
mind-to-mind, and mind-to computer. This fusion of human biology
with machines and cybernetic systems makes coding experts
extremely valuable, and hackers extremely powerful. And one of the
emergent consequences of this intimately interconnected world is
that hackers have found ways to implant false memories in people’s
minds, altering who they think they are.
This possibility for mind and memory manipulation gets to the
heart of Ghost. Beneath the movie’s visually stunning graphics and
compelling sci-fi storyline (as you may gather, I really like this
movie), Ghost in the Shell challenges us to think about what it means
to be alive, to have value, and to have a sense of self, purpose, and
destiny. On the release of the Ghost in the Shell remake in 2017 (a
poor “ghost” of a movie in comparison), commentator Emily Yoshida
described the original as a “meditation on consciousness and the
Ghost in a Shell: Being Human in an Augmented Future
focused on fixing what they see as the limitations of our biological
selves, the boundaries between biology, machines, and cyberspace
are becoming increasingly blurred.
philosophy of the self.”[^86] And she’s spot on. Just as Never Let Me Go
in chapter three forces viewers to think about what it means to be
human, Ghost takes us on a journey of contemplation around what it
means to be a conscious and self-aware entity, in a future where the
biological origins of humanity have increasingly less meaning.
At the center of Ghost is Major Motoko Kusanagi (voiced by Atsuko
Tanaka). Motoko is part of an elite team in “Section 9”—a shady
government department that operates at the edge of the law to
keep the wheels of society turning smoothly. Major Kusanagi is a
cyborg. Most of her body has been replaced by manufactured parts,
including much of her brain (although she retains a small part of
her original biological brain). She is strong, fast, cyber-connected,
and with the use of advanced “thermoptic technology” built into
her artificial skin, she is able to blend into her surroundings and
effectively disappear. She is also very human in her hopes, fears,
feelings, and relationships.
At the beginning of the movie, we learn that an aide to a senior
diplomat has been “ghost-hacked.” Her neural implant has been
used to hack into her mind, with the intent of using her to interfere
with a sensitive international negotiation. The hacking is traced to
a garbage collector who, we learn, believes (incorrectly) that he is
hacking into his wife’s “ghost” to find out why their relationship
is on the rocks. And he in turn is being handled by a figure who
believes (wrongly) he is an agent working a foreign government.
We quickly gather that the neural implants most people have allow
smart hackers to alter their sense of their own identity, or their
“ghost.” They can, in effect, rewrite who someone thinks they are.
And so it turns out that the garbage collector has no wife or family,
but lives alone with his dog. And the foreign agent has no idea of
who he really is. Rather, each has been manipulated by a shady
master-hacker called the Puppet Master.
This plays deeply into Major Kusanagi’s personal angst. She’s already
grappling with her own self-identity, and this ability for someone to
alter another person’s sense of self worries her. As a result, she is
deeply concerned about whether she’s who she thinks she is, and if
her sense of self is simply an illusion created by someone else. This
all adds to her uncertainty around what gives someone like herself
legitimacy, or worth, and what—if anything—makes her more than
just a machine?
In the movie, we repeatedly find Motoko deep in contemplation,
exploring her own mortality, and wrestling with who she is. There’s
one beautiful transition scene, for instance, where through a
masterful combination of visuals and music, we’re invited to share in
Motoko’s introspection. Motoko knows that she is largely made up
of manufactured parts, and that she may not be who she thinks she
is. But how does she make sense of this, and come to terms with it?
In the movie, there are two parallel narratives that weave together
through this introspection. Early on, we learn that a new recruit
to Section 9—Togusa (Kôichi Yamadera)—is the only member of
the team without implants. When he asks Major Kusanagi why he
was selected, she points out that overspecialization leads to death,
and that diversity of ability and perspective is essential for life.
This theme of diversity recurs at the movie’s denouement. But it
also underlies a meditation that threads through the movie on the
importance of embracing difference.
The second narrative is subtler, and it revolves around feelings
of friendship and love between Motoko and her colleague Batou
(voiced by Akio Ôtsuka). Despite Motoko’s crisis of self-identity,
it’s clear through the movie that Batou cares deeply for her. This
is a relationship that transcends who made their bodies, and how
“biological” they are; it invites us as viewers to think about what
the basis of this friendship is. The answer, it emerges, lies in the
“ghosts” that define both Motoko and Batou, and is not constrained
by physical form. There’s an essence within each of these characters
that transcends their physical bodies, and leads to a strong bond
between them. Yet it also extends to their physical interactions in
unexpected ways. In the movie, Batou is touchingly sensitive to
protecting Motoko’s dignity. This being Japanese science fiction
anime, there’s a fair amount of female nakedness, aided by Major
Kusanagi’s need to remove her clothes to take advantage of her
thermoptic skin. Yet we repeatedly find Batou averting his eyes from
Kusanagi’s naked body, and covering her nakedness where he can.
There is a sensitivity to his body language here that makes little
sense in the context of Motoko being a machine, but much sense
in terms of her being someone he has deep regard for. This regard
threads through the movie to its end, where Batou saves Motoko’s
Ghost in a Shell: Being Human in an Augmented Future
These ideas echo many of those touched on in movies like Never
Let Me Go (chapter three), Minority Report (chapter four) and Ex
Machina (chapter eight). But in Ghost, they are front and center of
this meditation that’s masquerading as an anime movie.
life. It’s a relationship that’s based on respect, acceptance, and
empowerment, even as Motoko is transformed into something other
than what she started as.
Returning to the plot, following the attempted hack of the diplomat’s
aide, the hunt is on for the Puppet Master. Another government
agency—Section 6—sets the cyber-equivalent of a honey trap for
the Puppet Master by creating a cyber-body/brain that he/she will
find irresistible to hack and download themselves into. The trap
is sprung, but the body containing the Puppet Master escapes the
facility it was being held in. However, its freedom is short-lived,
as it’s hit by a truck, and the mangled cybernetic body ends up in
the hands of Section 9. And this is where we begin to discover that
things are not quite as they seem.
It turns out that the Puppet Master (voiced by Iemasa Kayumi) is an
algorithm—codenamed project 2501—designed to hack people and
cyber-systems and manipulate them. The creators of 2501 thought
they had it under control. But the algorithm became self-aware and
escaped out into the net. And Section 6 has been trying to capture it
ever since.
As 2501 learned more of the world it found itself in, it became
aware of its own limitations, and especially its inability to do the
two things it deduced were essential to the growth of a species:
to reproduce, while adding diversity to the cyber-equivalent of the
gene pool, and to die, thus paving the way for new entities to grow,
mature, and evolve.
At this point, the movie begins to dive deeply into exploring the
meaning of life, and the roles and responsibilities of individuals
within a self-aware society. From 2501’s perspective, reproduction
through copying itself would be meaningless, a sterile act, and a
negation of what it considers to be meaningful. Instead, it begins
to explore how it can increase diversity within future generations
of the life form it represents, and to make way for these future
generations by experiencing death[^87].
Here, Major Kusanagi becomes central to 2501’s plan. In Kusanagi,
2501 sees an entity that is close enough to himself/herself[^88] for a
Once there, 2501 requests political asylum as a life-form. But Section 6 aren’t having any of this; they simply want their algorithm back. And so, Section 6 operatives carry out a raid to regain possession of the cyber-body holding 2501. They succeed in abducting him/her, but not before 2501 has intrigued Motoko enough for her to want find out more. Motoko chases after 2501’s abductors, and ends up in a deserted warehouse, with minimal backup, and an autonomous tank protecting her quarry. After a firefight where Major Kusanagi is heavily out-gunned (but not outsmarted), and where, in a very in-your-face metaphor, a wall carving of the evolutionary tree of life is shot up, Motoko reaches the tank. In her attempt to disable it and protect 2501, she compromises her cybernetic body, sacrificing her physical self in her quest for enlightenment. At this point, Batou arrives and saves both Motoko and 2501, but not before their physical bodies have been badly damaged. Thankfully, their minds are still intact, and in the few minutes they have together, 2501 and Motoko connect. This is where we learn that this union has been 2501’s plan all along—not to hack Motoko, but to engage with her as an equal. 2501 explains his/her fears and aspirations, and presents Motoko with a proposal: that they cybernetically merge, and in the process, create a new, more diverse, and richer entity, while allowing 2501 in his/her current form to die. Motoko agrees, and the merge takes place. Batou escapes with Motoko/2501’s intact head, and finds a replacement cyber body for this new entity. As the movie closes, the merging of 2501 and Motoko affirms that embracing the future, while letting go of the past, is essential for growth. By letting go of their individual identities and embracing diversity, Motoko and 2501 have, together, formed a more confident and self-assured life-form. And despite the “evolution” of Major Kusanagi, Batou’s respect and regard are not in the slightest diminished as he accepts this transformation within his friend. The underlying messages here may all sound a little pop psychology-ish. But despite this, Ghost helps peel the layers away
Ghost in a Shell: Being Human in an Augmented Future
bond to be developed, and procreation to occur. And so, to engineer
a situation where he/she and Kusanagi can interface, 2501 sets in
motion a series of events that lead to her/him being picked up by
Section 9.
from increasing tough questions around who we are and how we
interact with others, as emerging technological capabilities take us
increasingly beyond the limits of our biological evolution.
In July 2012, Dr. Steve Mann was allegedly assaulted in a Paris
branch of McDonald’s.[^89] What made this case unusual was that the
assault was sparked by a computer vision system physically attached
to Mann’s skull—a physical augmentation that others purportedly
took exception to.
Mann developed his “EyeTap” in 1999 as a computer-augmented
extension of his eye, allowing him to both record what he was
seeing and project information directly into his right eye. In many
ways, it was a precursor to Google Glass, but with one important
difference: the EyeTap was physically attached to his head, and
could not be removed without special tools.
In the incident that Mann described on his blog, a McDonald’s
employee attempted to physically pull the EyeTap off his head,
damaging it in the process, and causing considerable personal
distress. While the details of the case remain uncertain, it stands as
one of the first documented incidences of possible discrimination
against someone with an intentional body augmentation that,
because of its nature, led to a perceived threat to someone else;
although in this case, whether that perceived threat was to privacy,
“normalcy,” or something else, is unclear.
Mann’s use of technological augmentation is part of a broader “body
hacking” movement—a loose trend where people are experimenting
with do-it-yourself body enhancements. Many of these hacks
involve individuals embedding magnets in their bodies so they can
sense and respond to magnetic fields, or inserting radio frequency
identification (RFID) chips under their skin so they can remotely
interact with their environment. But in this extension of the maker
movement, people are playing with increasingly sophisticated ways
to incorporate novel technologies in their bodies, often through
unsupervised do-it-yourself surgery.
The ethics of untrained and unsupervised people cutting themselves
and others open to insert objects of unknown provenance are
To some at least, this is seen as part of our evolutionary
development (although it should be said that it’s a stretch to think
that using our intellect to merge our bodies with machines is directly
equatable to biological natural selection). Body hackers are often
enamored with the idea that we can use technology to overcome
our biological limitations, and transcend our evolutionary heritage to
become something else entirely. To many of them, placing magnets
and RFIDs under the skin are baby steps to something much greater:
becoming “trans-human.”
In recent years, the transhumanist movement has blossomed. As
technological capabilities have continued to grow and converge in
areas as diverse as robotics, nanotechnology, AI, neurotechnology,
and biotechnology, a growing number of people have become
enamored with the ability of technology to transform who we are,
and what we can achieve as a result. Prominent transhumanists such
as Ray Kurzweil and Nick Bostrom talk about enhancing physical
and mental abilities through technology, extending lifespans,
interfacing ever more deeply with computers, and one day even
leaving our biological bodies altogether. In the 2016 US election,
there was even a transhumanist candidate—Zoltan Istvan.[^90] As
I’m writing this, he’s setting his sights on becoming the Governor
of California.
Without doubt, an increasing ability to merge individuals with
powerful technologies opens up some compelling possibilities.
We’re already seeing this in some of the incredibly sophisticated
robotic and cyber-enabled medical devices and prosthetics that are
being developed. But these are just the tip of the iceberg compared
to what could be possible over the next decade or so. Advances
in AI-related technologies, computing architectures, gene editing
and manipulation, robotics, on-demand additive manufacturing,
and the converging and merging of these and other technologies,
Ghost in a Shell: Being Human in an Augmented Future
interesting to say the least, never mind the safety concerns.
However, this movement provides some indications as to where
human enhancement may be heading, and some of the bumps in
the road that it may encounter on the way. It’s also an early step
toward a future that echoes the one we’re introduced to in Ghost
in the Shell, where the lines are increasingly blurred between our
biological and our technological selves.
is massively accelerating what is possible. And while I’m skeptical
of technologies like Elon Musk’s neural lace becoming a reality any
time soon, we’re not as far as we sometimes think from technologies
that will make us faster, stronger, smarter, healthier, and capable of
doing things we never dreamt possible.
Yet these emerging technological capabilities come with a complex
array of risks, as Steve Mann’s experience showed. As a species, we
are embarrassingly programmed to see “different” as “threatening,”
and to take instinctive action against it. It’s a trait that’s exploited
in many science fiction novels and movies, including those in
this book. If we want to see the rise of increasingly augmented
individuals, we need to be prepared for some social strife.
We’re also going to have to grapple, perhaps more than in any
previous technological age, with what it means to be “human” as we
artificially augment ourselves.
In 2012, Oscar Pistorius made history by being the first runner
to compete in the Olympic Games with two prosthetic legs. Even
for those not glued to the event, his iconic racing blades came to
represent the promise of technological enhancements to overcome
human limitations. Yet they also stirred up a controversy: Did
Pistorius’ prosthetics give him an unfair advantage? Did they
somehow make him “more than” his fellow competitors? Sadly,
Pistorius went on to prove just how human he was, and in
December 2015 was convicted of the murder of his girlfriend Reeva
Steenkamp. But the story of his blades is nevertheless one that
challenges how we think about using technology to change and
extend our innate abilities.
Pistorius was born with a congenital absence of the fibula, and at
eleven months old, his legs were amputated below the knee. Despite
this, he developed into a strong and competitive sportsperson, and
in the mid-2000s began making a splash running on “blades”—
blade-like prosthetic lower legs, designed specifically for the track.
But this wasn’t the first time the world had seen such an unusual
body augmentation.
Blades were the brainchild of Van Phillips, an American inventor
who lost one of his legs below the knee when he was twentyone. Phillips wanted to create a prosthetic foot that did more than
replicate a human foot. Using a cheetah’s hind legs as inspiration,
Early on, Phillips worked with another double amputee, the sprinter,
actor, and model Aimee Mullins. Mullins wowed the world with her
“cheetah” legs in a 1998 TED Talk[^91] that reputedly cemented the
TED brand. She repeated the “wowing” in 2009 with her TED Talk
“My Twelve Pairs of Legs,”[^92] where she introduced her audience
to the idea that, far from correcting a disability, prosthetics can be
transformative. As she concludes in that talk:
That’s when I knew that the conversation with society has
changed profoundly in this last decade. It is no longer
a conversation about overcoming deficiency. It’s a
conversation about augmentation. It’s a conversation about
potential. A prosthetic limb doesn’t represent the need to
replace loss anymore. It can stand as a symbol that the
wearer has the power to create whatever it is that they
want to create in that space.
Mullins’s vision was one of vast potential, as machines and
cybernetics are increasingly engineered together to extend human
performance. But this same potential was to become a thorn in
Pistorius’s side in the hyper-conservative world of international
sport. And at the tip of that thorn was the nagging worry that his
blades somehow gave him a competitive advantage. Even as the
world was beginning to accept that someone labeled as “disabled”
could compete in mainstream sport, society was working hard to
ensure that these “others” didn’t out-perform “normal” competitors.
Following concerns that blades and similar devices could give
runners a competitive advantage, in 2007 the International
Association of Athletics Federation (IAAF) banned the use of “any
technical device that incorporates springs, wheels or any other
element that provides a user with an advantage over another athlete
Ghost in a Shell: Being Human in an Augmented Future
he created a leg/foot combination that worked like a spring, storing
energy when it hit the ground, and propelling the leg forward.
Phillips started his company Flex-Foot Incorporated in 1984, and
continued to work on refining the design for some time after that.
not using such a device.”[^93] In fact, so great was the paranoia over
Pistorius’ prosthetics that the IAAF monitored his performance
to see if they could detect any signs of an advantage, and they
supported research to the same end. In 2008, they concluded that
the blades he was using allowed him to perform better than nonaugmented runners, rendering them ineligible for competitions,
including the 2008 Olympics.
Later research indicated that things were more complex than this,
and in 2012, Pistorius was allowed to compete in the London
Olympics. You could almost hear the IAAF breathe a collective
sigh of relief when he didn’t win. By this time, though, it was clear
that the merest hint of mechanical body enhancements allowing
someone to perform a hair’s breadth better than non-enhanced
competitors was anathema to the sports world.
Both Pistorius’s and Mullins’s stories fascinate me as, they reveal
two very different sides of societal attitudes toward human
augmentation. On one hand, we have Mullins’s infectious enthusiasm
over how her prosthetic legs increase her versatility. They become
an extension of her self-expression, and a tool to extend her
capabilities. Hers is a narrative of self-expression and personal
achievement that inspires us, but doesn’t threaten us.
On the other hand, we have Pistorius’s fight with the IAAF for
acceptance and legitimacy, precisely because his augmentation
was seen as a threat. As Pistorius rose in fame and ability, there
was a growing fear that he would best “normal” athletes, and
win through having an undue advantage. And here we see a
convergence between the two stories. As a species, we’re remarkably
good at celebrating success, as long as it doesn’t undermine our
sense of how the world should be. But as soon as our worldview
comes under threat, we dig in. And this is where we hit the sharp
end of what will inevitably become a growing debate around
cybernetic augmentation.
Mullins, Pistorius, and others using advanced prosthetics are a
long way removed from the augmentations in Ghost in the Shell.
Nevertheless, they do foreshadow a future where what defines
“normal,” and by extension, what defines “human,” becomes
Here, I’m using “normal” intentionally and provocatively, as at
the center of this challenge is our built-in social survival instinct
of grouping together and isolating anyone, or anything, that is
perceived to be threateningly not-normal. Socially, we’re remarkably
good at being open-minded and accepting of diversity when it’s not
seen as a threat. But as soon as enough people perceive “different”
as threatening something they value, whether it’s their lifestyle, their
possessions, their beliefs or their identity, there is a metaphorical
circling of the wagons. Through history we’ve seen this with race,
gender, socioeconomic status, appearance, character, beliefs, political
affiliation, and pretty much anything that can be labeled as defining
someone as being different from the crowd. It’s not a pleasant
human trait. But it is one that kicks in when we’re content to go
with the social flow and stop thinking. And it’s going to be an issue
when it comes to body augmentations that threaten the status quo.
But it gets worse. There’s an easy shorthand that people slip into
when what they consider to be “normal” is threatened, and this
involves implicitly equating the divide between “normal” and
“abnormal” with “human” and “not human,” just as we saw with
Never Let Me Go in chapter three. Few people, I suspect, would
admit that they think of people who they perceive as threatening as
not being quite human. But the narrative’s there nevertheless. Just
look at the language that’s been used over the centuries to denigrate
people of color, or people of other races, people of other religions,
people who are intellectually, emotionally and physically different
from “the norm,” and people with non-binary gender identities.
There’s a dark, deep tendency to label threateningly different
traits and abilities as “non-human” or even “sub-human” in our
collective psyche.
This will inevitably become more of a social issue as technologies
advance to the point where we can use augmentation to enhance
human abilities beyond what is considered normal. But it will
also become increasingly important for the self-identity and selfacceptance of those who have enhanced abilities. This, again, is not
Ghost in a Shell: Being Human in an Augmented Future
increasingly important. This echoes the challenges of cognitive
enhancement seen with Limitless (chapter five) and the human
cloning in Never Let Me Go (chapter three). And it emphasizes a
particularly knotty challenge that the body-hacking movement also
highlights: How do we navigate a future where technology not only
has the capacity to bring everyone to “normal” spec, but also to
redefine what “normal” means in the first place?
a new narrative. Labeling someone as “inferior” or “less worthy”—
both subtle metaphors for “not quite as human as the rest of us”—
can engender self-doubt that is ultimately deeply debilitating. But
such labeling also sets up tensions that can lead to tipping points in
the social fabric and bring about revolutions—whether cultural or
physical, or both—that lead to a readjustment of what is considered
normal and what is not. This is sometimes necessary as society
grows and evolves. But sometimes these transitions are deeply
damaging in ways that could be avoided.
As augmentation technologies continue to advance, we’re going
to have to grapple with how to evolve as a society without falling
prey to our instincts to deprecate the value of those we perceive
as threatening us. This will require developing a society-wide
appreciation of the perceived and actual risks and benefits of
augmentation and enhancement. And it’ll take plenty of soulsearching around our collective values, and how we put them
into practice.
The good news is that we already have a long history of
augmentation that helps set the baseline for future advances.
People augment their eyesight with glasses, contact lenses, and eye
surgery. The clothes we wear augment how we express and define
ourselves us. Our computers, phones, and other devices augment
us by connecting us to vast and powerful networks. And medical
devices, from pacemakers to replacement body parts, augment us by
extending our ability to live healthy, fulfilled lives. We are, without a
doubt, already a technologically augmented and enhanced species.
Yet we’ve assimilated these augmentations in ways that lead to their
acceptance when they don’t confer what we consider to be an unfair
advantage, and that question them where they threaten something
we consider important. This is human instinct, and an evolved
survival mechanism. But it’s also socially lazy. It’s an assimilation
that lacks consideration and intentionality, and it’s one that’s not
strongly guided by moral values and ideals. And because of this, it’s
an assimilation that can appear enlightened until a serious perceived
threat appears, at which point instinct takes over with a vengeance.
If we’re going to ensure the beneficial, equitable, and—let’s be
honest, life-enhancing and affirming—development of augmentation
technologies, we’re going to have to get a lot better as a society at
working out what’s important, and intentionally opening pathways
for this to occur. And this is going to mean stepping away from
But this raises another challenge that Ghost in the Shell addresses
full-on: the possibility of our augmented selves being hacked by
others, especially when this augmentation extends to developing
ways of directly connecting our brains to machines.
The physical augmentations in Ghost in the Shell, including
Batou’s eyes and Motoko’s body, are important. But it’s the neural
augmentations that ultimately drive the narrative. In the metaphor
of the movie’s title, the physical body is merely a shell, whether it’s
augmented or not. This in turn houses the essence of what makes
someone who they are, and gives them their identity, their ghost.
Yet in the world of the movie, this “ghost” is vulnerable, precisely
because it depends on technological augmentation.
In Western culture, we deeply associate our brains with our identity.
They are the repository of the memories and the experiences that
define us. But they also represent the inscrutable neural circuits that
guide and determine our perspectives, our biases, our hopes and
dreams, our loves, our beliefs, and our fears. Our brain is where our
cognitive abilities reside (“gut” instinct not withstanding); it’s what
enables us to form bonds and connections with others, and it’s what
determines our capacity to be a functioning and valuable part of
society—or so our brains lead us to believe. To many people, these
are essential components of the cornucopia of attributes that define
them, and to lose them, or have them altered, would be to lose part
of themselves.
This is, admittedly, a somewhat skewed perspective. Modern
psychology and neurology are increasingly revealing the
complexities and subtleties of the human brain and the broader
biological systems it’s intimately intertwined with. Yet despite
this, for many of us, our internal identity—how we perceive and
Ghost in a Shell: Being Human in an Augmented Future
our instinctual fear of differences that we perceive as threatening,
and getting better at embracing diversity. At the same time, we’re
going to have to be intentional in how we develop and implement
the frameworks within which augmentation occurs, so that sociallyagreed-on values guide the use of augmentation technologies. And
as increasingly advanced technologies challenge embedded but
outmoded notions of what it is to be “human,” we’re going to have
to think hard about what we mean by personal value, worth, and
rights.
understand ourselves, and who we believe we are—is so precious
that anything that threatens it is perceived as a major risk. This is
why neurological diseases like Alzheimer’s can be so distressing,
and personality changes resulting from head traumas so disturbing.
It’s also why it can be so unsettling when we see people we know
undergoing changes in their personality or beliefs. These changes
force us to realize that our own identity is malleable, and that we in
turn could change. And, as a result, we face the realization that the
one thing we often rely on as being a fixed certainty, isn’t.
Over millennia, we’ve learned as a species to cope with the fragility
of self-identity. But this fragility doesn’t sit comfortably with us.
Rather, it can be extremely distressing, as we recognize that disease,
injuries, or persuasive influences can change us. As a society, we
succeed most of the time in absorbing this reality, and even in
some cases embracing it. But neural enhancements bring with them
a brand new set of threats to self-identity, and ones that I’m not
sure we’re fully equipped to address yet, including vulnerability to
outside manipulation.
Elon Musk’s neural lace is a case in point, as a technology with
both vast potential and largely unknown risks. It’s easy to imagine
how overlaying the human brain with a network of connections,
processors and communications devices could vastly enhance our
abilities and allow us to express ourselves more completely. Imagine
if you could control your surroundings through your thoughts.
Or you could type, or search the net, just by thinking about it. Or
even if you could turbocharge your cognitive abilities at the virtual
press of a button, or change your mood, recall information faster,
get real-time feedback on who you’re speaking with, save and
recall experiences, manipulate vast cyber networks, all through the
power of your mind. It would be like squeezing every technological
advancement from the past five hundred years into your head, and
magnifying it a hundred-fold. If technologies like the neural lace
reached their full potential, they would provide an opportunity for
users to far exceed their full biological potential, and express their
self-identity more completely than ever before.
It’s not hard to see how seductive some people might find such
a technology. Of course, we’re a long, long way from any of this.
Despite massive research initiatives on the brain, we’re still far
from understanding the basics of how it operates, and how we can
manipulate this. Yet this is not stopping people from experimenting,
despite what this might lead to.
In this brief science fiction story, Kennedy, a.k.a. Alpha O. Royal,
describes a future where brains can be disconnected from their
bodies, and people can inhabit a virtual world created by sensors
and probes that directly read and stimulate their neurons. In the
book, this becomes the key that opens up interplanetary travel,
as hurling a wired-up brain through space turns out to be a lot
easier than having to accompany it with a body full of inconvenient
organs. Fantastical as the book is, Kennedy uses it to articulate
his belief that the future of humanity will depend on connecting
our brains to the wider world through increasingly sophisticated
technologies; starting with his hollow brain probes, and extending
out to wireless-linked probes, that are able to read and control
neurons via light pulses.
Amazingly, we are already moving closer to some of the sensing
technology that Kennedy envisions in 2051. In 2016, researchers
at the University of California, Berkeley announced they had built
a millimeter-sized wireless neural sensor that they dubbed “neural
dust.” Small numbers of these, it was envisaged, could be implanted
in someone’s head to provide wireless feedback on neural activity
from specific parts of the brain. The idea of neural dust is still at
a very early stage of development, but it’s not beyond the realm
of reason that these sensors could one day be developed into
sophisticated wireless brain interfaces.[^96] And so, while Kennedy’s
Ghost in a Shell: Being Human in an Augmented Future
In 2014, the neurosurgeon Phil Kennedy underwent elective brain
surgery, not to correct a problem, but in an attempt to create
a surgically implanted brain-machine interface.[^94] Kennedy had
developed a deep brain probe that overcame the limitations of
simply placing a wire in someone’s brain, by encouraging neurons
to grow into a hollow glass tube. By experimenting on himself, he
hoped to gain insight into how the parts of the brain associated
with language operate, and whether he could decode neural signals
as words. But he also had a vision of a future where our brains are
intimately connected to machines, one that he captured in the 2012
novel 2051, published under the pseudonym Alpha O. Royal.[^95]
sci-fi story stretches credulity, reality isn’t as far behind as we
might think.
There’s another side of Kennedy’s story that is relevant here, though.
2051 is set in a future where artificial intelligence and “nanobots”
(which we’ll reencounter in chapter nine) have become a major
threat. In an admittedly rather silly plotline, we learn that the
real-life futurist and transhumanist Ray Kurzweil has loaned the
Chinese nanobots which combine advanced artificial intelligence
with the ability to self-replicate. These proceed to take over China
and threaten the rest of the world. And they have the ability to hack
into and manipulate wired-up brains. Because everything that these
brains experience comes through their computer connections, the AI
nanobots can effectively manipulate someone’s reality with ease, and
even create an alternate reality that they are incapable of perceiving
as not being real.
The twist in Kennedy’s tale is that the fictitious nanobots simply
want global peace and universal happiness. And the logical route
to achieving this, according to their AI hive-mind, is to assimilate
humans, and convince them to become part of the bigger collective.
It’s all rather Borg-like if you’re a Start Trek fan, but with a
benevolent twist.
Kennedy’s story is, admittedly, rather fanciful. But he does hit on
what is probably one of the most challenging aspects of having a
fully connected brain, especially in a world where we are seceding
increasing power to autonomous systems: vulnerability to hacking.
Some time ago, I was speaking with a senior executive at IBM, and
he confessed that, from his elevated perspective, cybersecurity is
one of the greatest challenges we face as a global society. As we see
the emergence of increasingly clever hacks on increasingly powerful
connected systems, it’s not hard to see why.
Cyberspace—the sum total of our computers, the networks they
form, and the virtual world they represent—is unique in that it’s
a completely human-created dimension that sits on top of our
reality (a concept we come back to in chapter nine and the movie
Transcendence). We have manufactured an environment that quite
literally did not exist until relatively recently. It’s one where we can
now build virtual realities that surpass our wildest dreams. And
because, in the early days of computing, we were more interested
Of course, the digital community learned early on that cybersecurity
demanded at least as much attention to good practices, robust
protocols, smart design, and effective governance as any physical
environment, if people weren’t going to get hurt. But certainly, in
the early days, this was seasoned with the idea that, if everything
went pear-shaped, someone could always just pull the plug.
Nowadays, as the world of cyber is inextricably intertwined with
biological and physical reality, this pulling-the-plug concept seems
like a quaint and hopelessly outmoded idea. Cutting off the power
simply isn’t an option when our water, electricity, and food supplies
depend on cyber-systems, when medical devices and life-support
systems rely on internet connectivity, where cars, trucks and other
vehicles cannot operate without being connected, and where
financial systems are utterly dependent on the virtual cyber worlds
we’ve created.
It’s this convergence between cyber and physical realities that
is massively accelerating current technological progress. But it
also means that cyber-vulnerabilities have sometimes startling
real-world consequences, including making everything from
connected thermostats to digital pacemakers vulnerable to attack
and manipulation. And, not surprisingly, this includes brainmachine interfaces.
In Ghost in the Shell, this vulnerability leads to ghost hacking,
the idea that if you connect your memories, thoughts, and brain
functions to the net, someone can use that connection to manipulate
and change them. It’s a frightening idea that, in our eagerness to
connect our very soul to the net, we risk losing ourselves, or worse,
becoming someone else’s puppet. It’s this vulnerability that pushes
Major Kusanagi to worry about her identity, and to wonder if she’s
already been compromised, or whether she would even know if she
had been. For all she knows, she is simply someone else’s puppet,
being made to believe that she’s her own person.
With today’s neural technologies, this is a far-fetched fear. But still,
there is near-certainty that, if and when someone connects a part
of their brain to the net, someone else will work out how to hack
in what we could do rather than what we should (or even how we
should do it), this environment is fraught with vulnerabilities. Not to
put too fine a point on it, we’ve essentially built a fifth dimension to
exist in, while making up the rules along the way, and not worrying
too much about what could go wrong until it was too late.
that connection. This is a risk that far transcends the biological
harms that brain implants and neural nets could cause, potentially
severe as these are. But there’s perhaps an even greater risk here.
As we move closer to merging the biological world we live in with
the cyber world we’ve created, we’re going to have to grapple with
living in a world that hasn’t had billions of years of natural selection
for the kinks to be ironed out, and that reflects all the limitations
and biases and illusions that come with human hubris. This is a
world wherein human-made monsters lie waiting for us to stumble
on them. And if we’re not careful, we’ll be giving people a one-way
neurological door into it.
Not that I think this should be taken as an excuse not to build brainmachine interfaces. And in reality, it would be hard to resist the
technological impetus pushing us in this direction. But at the very
least, we should be working with maps that says in big bold letters,
“Here be monsters.” And one of the “monsters” we’re going to face
is the question of who has ultimate control over the enhanced and
augmented bodies of the future.
If you have a body augmentation or an implant, who owns it? And
who ultimately has control over it? It turns out that if you purchase
and have installed a pacemaker or implantable cardiovascular
defibrillator, or an artificial heart or other life-giving and life-saving
devices, who can do what with it isn’t as straightforward as you
might imagine. As a result, augmentation technologies like these
raise a really tricky question—as you incorporate more tech into
your body, who owns you? We’re still a long way from the body
augmentations seen in Ghost in the Shell, but the movie nevertheless
foreshadows questions that are going to become increasingly
important as we continue to replace parts of our bodies with
machines.
In Ghost, Major Kusanagi’s body, her vital organs, and most of her
brain are manufactured by the company Megatech. She’s still an
autonomous person, with what we assume is some set of basic
human rights. But her body is not her own. Talking with her
colleague Batou, they reflect that, if she were to leave Section 9, she
would need to leave most of her body behind. Despite the illusion
of freedom, Kusanagi is effectively in indentured servitude to
someone else by virtue of the technology she is constructed from.
In 2015, Hugo Campos wrote an article for the online magazine
Slate with the sub-heading, “I can’t access the data generated by
my implanted defibrillator. That’s absurd.”[^97] Campos had a device
inserted into his body—an Implantable Cardiac Defibrillator, or
ICD—that constantly monitored his heartbeat, and that would
jump-start his heart, were it to falter. Every seven years or so,
the implanted device’s battery runs low, and the ICD needs to be
replaced, what’s referred to as a “generator changeout.” As Campos
describes, many users of ICDs use this as an opportunity to upgrade
to the latest model. And in his case, he was looking for something
specific with the changeout; an ICD that would allow him to
personally monitor his own heart.
This should have been easy. ICDs are internet-connected these days,
and regularly send the data they’ve collected to healthcare providers.
Yet patients are not allowed access to this data, even though it’s
generated by their own body. Campos’ solution was to purchase an
ICD programmer off eBay and teach himself how to use it. He took
the risk of flying close to the edge of legality to get access to his
own medical implant.
Campos’ experience foreshadows the control and ownership
challenges that increasingly sophisticated implants and cyber/
machine augmentations raise. As he points out, “Implants are
the most personal of personal devices. When they become an
integral part of our organic body, they also become an intimate
part of our identity.” And by extension, without their ethical
and socially responsive development and use, a user’s identity
becomes connected to those that have control over the device and
its operations.
Ghost in a Shell: Being Human in an Augmented Future
Even assuming that there are ethical rules against body
repossession, Kusanagi is dependent on regular maintenance
and upgrades. Miss a service, and she runs the risk of her body
beginning to malfunction, or becoming vulnerable to hacks and
attacks. In other words, her freedom is deeply constrained by the
company that owns her body and the substrate within which her
mind resides.
In the case of ICDs, manufacturers and healthcare providers still
have control over the data collected and generated by the device.
You may own the ICD, but you have to take on trust what you are
told about the state of your health. And you are still beholden to
the “installers” for regular maintenance. Once the battery begins to
fail, there are only so many places you can go for a refit. And unlike
a car or a computer, the consequence of not having the device
serviced or upgraded is possible death. It’s almost like being locked
into a phone contract where you have the freedom to leave at any
time, but contract “termination” comes with more sinister overtones.
Almost, but not quite, as it’s not entirely clear if users of ICDs even
have the option to terminate their contracts.
In 2007, Ruth and Tim England and John Coggins grappled with
this dilemma through the hypothetical case of an ICD in a patient
with terminal cancer.[^98] The hypothetical they set up was to ask
who has the right to deactivate the device, if constant revival in
the case of heart failure leads to continued patient distress. The
scenario challenges readers of their work to think about the ethics
of patient control over such implants, and the degree of control that
others should have. Here, things turn out to be murkier than you
might think. Depending on how the device is classified, whether it
is considered a fully integrated part of the body, for instance, or an
ongoing medical intervention, there are legal ramifications to who
does what, and how. If, for instance, an ICD is considered simply
as an ongoing medical treatment, the healthcare provider is able to
decide on its continued use or termination, based on their medical
judgment, even if this is against the wishes of the patient. In other
words, the patient may own the ICD, but they have no control over
its use, and how this impacts them.
On the other hand, if the device is considered to be as fully
integrated into the body as, say, the heart itself, a physician will have
no more right to permanently switch it off than they have the right
to terminally remove the heart. Similarly, the patient does not legally
have the right to tamper with it in a way that will lead to death, any
more than they could legally kill themselves.
In this case, England and colleagues suggest that intimately
implanted devices should be treated as a new category of medical
England’s work is helpful in thinking through some of the
complexities of body implant ethics. But it stops far short of
addressing two critical questions: Who has the right to access and
control augmentations designed to enhance performance (rather
than simply prevent death), and what happens when critical
upgrades or services are needed?
This is where we’re currently staring into an ethical and
moral vacuum. It might not seem such a big deal when most
integrated implants at the moment are health-protective rather
than performance-enhancing. But we’re teetering on the cusp
of technological advances that are likely to sweep us toward an
increasingly enhanced future, without a framework for thinking
about who controls what, and who ultimately owns who you are.
This is very clear in emerging plans for neural implants, whether
it’s Neuralink’s neural lace or other emerging technologies for
connecting your brain to the net. While these technologies will
inevitably have medical uses—especially in treating and managing
neurological diseases like Parkinson’s disease—the expectation
is that they will also be used to increase performance and ability
in healthy individuals. And as they are surgically implanted,
understanding who will have the power to shut them down, or to
change their behavior and performance, is important. As a user, will
you have any say in whether to accept an overnight upgrade, for
instance? What will your legal rights be when a buggy patch leads
to a quite-literal brain freeze? What happens when you’re given the
choice of paying for “Neuralink 2.0” or keeping an implant that is no
longer supported by the manufacturer? And what do you do when
you discover your neural lace has a hardware vulnerability that
makes it hackable?
This last question is not idle speculation. In August 2016, a report
from the short-selling firm Muddy Waters Capital LLC released a
report claiming that ICDs manufactured by St. Jude Medical, Inc.
Ghost in a Shell: Being Human in an Augmented Future
device. They refer to these as “integral devices” that, while not
organic, are nevertheless a part of the patient. They go on to suggest
that this definition, which lies somewhere between the options
usually considered for ICDs, will allow more autonomy on the part
of patient and healthcare provider. And specifically, they suggest that
“a patient should have the right to demand that his ICD be disabled,
even against medical advice.”
were vulnerable to potentially life-threatening cyberattacks.[^99] The
report claimed:
“We have seen demonstrations of two types of cyber-attacks
against [St Jude] implantable cardiac devices (‘cardiac devices’):
a ‘crash’ attack that causes cardiac devices to malfunction—
including by apparently pacing at a potentially dangerous rate;
and, a battery drain attack that could be particularly harmful
to device dependent users. Despite having no background in
cybersecurity, Muddy Waters has been able to replicate in-house
key exploits that help to enable these attacks.”
St. Jude vehemently denied the accusations, claiming that they were
aimed at manipulating the company’s value (the company’s stock
prices tumbled as the report was released). Less than a year later,
St. Jude was acquired by medical giant Abbott. But shortly after
this, hacking fears led to the US Food and Drug Administration
recalling nearly half a million former St. Jude pacemakers[^100] due to
an identified cybersecurity vulnerability.
Fortunately, there were no recorded cases of attacks in this instance,
and the fix was a readily implementable firmware update. But
the case illustrates just how vulnerable web-connected intimate
body enhancements can be, and how dependent users are on the
manufacturer. Obviously, such systems can be hardened against
attack. But the reality is that the only way to be completely cybersecure is to have no way to remotely connect to an implanted
device. And increasingly, this defeats the purpose for why a device
is, or might be, implanted in the first place.
As in the case of the St Jude pacemaker, there’s always the
possibility of remotely-applied patches, much like the security
patches that seem to pop up with annoying frequency on computer
operating systems. With future intimate body enhancements, there
will almost definitely be a continuing duty of care from suppliers to
customers to ensure their augmentations are secure. But this in turn
ties the user, and their enhanced body, closely to the provider, and it
leaves them vulnerable to control by the providing company. Again, the scenario is brought to mind of what happens when you, as an
enhanced customer, have the choice of keeping your enhancement’s
buggy, security-vulnerable software, or paying for the operating
system upgrade. The company may not own the hardware, but
without a doubt, they own you, or at least your health and security.
Things get even more complex as the hardware of implantable
devices becomes outdated, and wired-in security vulnerabilities are
discovered. On October 21, 2016, a series of distributed denial of
service (DDOS) attacks occurred around the world. Such attacks
use malware that hijacks computers and other devices and redirects
them to swamp cyber-targets with massive amounts of web traffic—
so much traffic that they effectively take their targets out. What
made the October 21 attacks different is that the hijacked devices
were internet-connected “dumb devices”: home routers, surveillance
cameras, and many others with a chip allowing them to be
connected to the internet, creating an “Internet of Things.” It turns
out that many of these devices, which are increasingly finding their
way into our lives, have hardware that is outdated and vulnerable
to being coopted by malware. And the only foolproof solution to
the problem is to physically replace millions—probably billions—
of chips.
The possibility of such vulnerabilities in biologically intimate devices
and augmentations places a whole new slant on the enhanced
body. If your enhancement provider has been so short-sighted as to
use attackable hardware, who’s responsible for its security, and for
physically replacing it if and when vulnerabilities are discovered?
This is already a challenge, although thankfully tough medical
device regulations have limited the extent of potential problems
here so far. Imagine, though, where we might be heading with
poorly-regulated innovation around body-implantable enhancements
that aren’t designed for medical reasons, but to enhance ability.
You may own the hardware, and you may have accepted any “buyer
beware” caveats it came with. But who effectively owns you, when
you discover that the hardware implanted in your legs, your chest,
or your brain, has to be physically upgraded, and you’re expected
to either pay the costs, or risk putting your life and well-being on
the line?
Without a doubt, as intimate body-enhancing technologies become
more accessible, and consumers begin to clamor after what (bio)tech companies are producing, regulations are going to have to change
and adapt to keep up. Hopefully this catch-up will include laws that
protect consumers’ quality of life for the duration of having machine
enhancements surgically attached or embedded. That said, there is
a real danger that, in the rush for short-term gratification, we’ll see
pushback against regulations that make it harder for consumers to
get the upgrades they crave, and more expensive for manufacturers
to produce them.
This is a situation where Ghost on the Shell provides what I suspect
is a deeply prescient foreshadowing of some of the legal and social
challenges we face over autonomy, as increasingly sophisticated
enhancements become available. The question is, will anyone pay
attention before we’re plunged into an existential crisis around who
we are, and who owns us?
One approach here is to focus less on changing ourselves, and
instead to focus on creating machines that can achieve what we only
dream of. But as we’ll see with the next movie, Ex Machina, this is a
pathway that also comes with its own challenges.
[^82]: @elonmusk, on Twitter, posted June 4, 2016 https://twitter.com/elonmusk/ status/739006012749799424
[^83]: Rolfe Winkler (2017) “Elon Musk Launches Neuralink to Connect Brains With Computers.” The Wall Street Journal, March 27, 2017. https://www.wsj.com/articles/elon-musk-launches-neuralink-toconnect-brains-with-computers-1490642652
[^84]: https://www.neuralink.com/ This was posted on the Neuralink home page as of October 9, 2017.
[^85]: “Fog computing” or “edge computing” uses a growing network of internet-connected devices to push data processing out of the cloud, and to the devices that are collecting and using information on everything from our personal habits to the environment around us. It’s the next iteration in distributed computing architectures that combines a vast array of relatively low-power devices with more centralized data processing to massively expand how large amounts of data are utilized.
[^86]: Emily Yoshida (2017) “A Beginner’s Guide to the Ghost in the Shell Universe” http://www.vulture. com/2017/03/a-beginners-guide-to-the-ghost-in-the-shell-series.html
[^87]: This emphasis in Ghost on death of the individual as an essential part of the growth across generations is especially intriguing, as it’s contrary to a lot of Western-style thinking that celebrates the ability of technology to prolong individual lives, possibly at the expense of future generations and social well-being.
[^88]: Although the physical manifestation of 2501 in the movie has sex-associated attributes, 2501 has no clear gender.
[^89]: You can read more about the details of this incident on Steve Mann’s blog. Steve Mann (2012) “Physical assault by McDonald’s for wearing Digital Eye Glass” Eyetap, posted July 16, 2012. http:// eyetap.blogspot.com/2012/07/physical-assault-by-mcdonalds-for.html
[^90]: You can read more about Zoltan Istvan’s aspirations and vision on his personal website: http:// www.zoltanistvan.com/
[^91]: Aimee Mullins (1998) “Changing my legs—and my mindset.” TED. https://www.ted.com/talks/ aimee_mullins_on_running
[^92]: Aimee Mullins (2009) “My 12 pairs of legs.” TED. https://www.ted.com/talks/aimee_mullins_ prosthetic_aesthetics
[^93]: The ruling by the IAAF, “IAAF Council introduces rule regarding ‘technical aids’” can be found on The Internet Archive, at https://web.archive.org/web/20080617001525/http://www.iaaf.org/news/ Kind%3D512/newsId%3D38127.html
[^94]: Daniel Engber provides a compelling account of Kennedy’s work in a 2016 Wired article titled “The Neurologist who Hacked His Brain, and Almost Lost His Mind.” Wired, January 26, 2016. https:// www.wired.com/2016/01/phil-kennedy-mind-control-computer/
[^95]: Alpha O. Royal (2012) “2051.” Available at Amazon.com.
[^96]: For more on neural dust sensors, see “Considering ethics now before radically new brain technologies get away from us.” Published on The Conversation, September 14 2016. https:// theconversation.com/considering-ethics-now-before-radically-new-brain-technologies-get-away-fromus-65215
[^97]: Hugo Campos (2015) “The Heart of the Matter,” published in Slate, March 24 2015. http://www. slate.com/articles/technology/future_tense/2015/03/patients_should_be_allowed_to_access_data_ generated_by_implanted_devices.html
[^98]: England, R., et al. (2007). “The ethical and legal implications of deactivating an implantable cardioverter-defibrillator in a patient with terminal cancer.” Journal of Medical Ethics 33(9): 538. http:// doi.org/10.1136/jme.2006.017657
[^99]: Muddy Waters Research report on St. Jude Medical, Inc. August 25, 2016. http://d. muddywatersresearch.com/research/stj/mw-is-short-stj/
[^100]: FDA, August 29, 2017. “Firmware Update to Address Cybersecurity Vulnerabilities Identified in Abbott’s (formerly St. Jude Medical’s) Implantable Cardiac Pacemakers: FDA Safety Communication.” https://www.fda.gov/medicaldevices/safety/alertsandnotices/ucm573669.htm