Hidden Layers: Spell Maps, Illusions, and Neural Networks

Anyone who’s ever watched Serial Experiments: Lain remembers the scene when Lain goes to greet her friends at school, but instead a doppelganger detaches from her and goes in her place–it’s the perfect expression of alienation, and evokes the idea that someone else is living your life. It also brings up questions about reality and identity: can we trust our senses to tell us what’s there or not? How many other things lie beyond sensory perception? Could someone fabricate reality? Are we who we think we are?

I like the idea of doppelgangers, but I like the idea of creating illusions even more. In Ursula LeGuin’s Wizard of Earthsea series, illusions are the easiest thing in the world, but when I sat down to figure out how to create illusions using my own system’s logic, I realized there were all kinds of difficulties: how do you trick all five senses? What sustains the illusion? What happens when you need to create something new, something that has never existed? The goal I had in mind was to create an illusory doppelganger, a kind of puppet that could be controlled by an unseen mage nearby.

Part of the process of creating an illusory person could be accomplished with an AutoCAD-like process–sculpting a person like a 3D model in isolation, adding details and textures like a video game character. But from there you run into the same problems video game characters do: how does the doppelganger ‘model’ interact with it’s environment realistically? How do you simulate the flapping of clothing when running, or when it’s windy out? How do you correctly recreate the sounds of footsteps on tile floors vs. cobblestones?How does the model deal with gravity and changes of elevation in terrain, let alone clipping through things like doors or tables? The model would need to be meticulously crafted to keep it from coming across as a glitchy mess.

The other issue is how to make it move, walk, and talk at all. One solution is to have its controller act out the movements verbatim in real-time with the situation, but that leads to all kinds of problems–if the controller is hidden somewhere, observing the situation, they need to react perfectly in time with outside actions, meaning that if there’s any disruption in line of sight, the whole facade is ruined. The biggest problem is when people or objects try to interact with the illusion model–if someone throws an apple at the doppelganger, the apple will pass through the illusion. Even if a controller were able to weave more illusions on the fly to correct this, by say, making an illusion that the model had caught the apple, the real apple would still make a sound as it hit the floor. The latency issues would be rough.

So there are a lot of issues here, and ones I didn’t really know how to solve practically. Luckily, Google came to the rescue.

One of the big recent announcements from Google’s I/O conference was that developers had created a method called AutoML, which is a system that guides artificial neural networks in creating other neural networks for a specific purpose, like speech or image recognition. Some of the networks created using AutoML actually surpassed the ones created by humans–meaning that an artificially intelligent system had beaten humans at creating systems similar to itself. What really caught my attention, though, was the structure of neural networks:

Image result for neural network structure

The nodes and interconnecting lines reminded me of summoning circles and occult diagrams, like the Tree of Sephiroth:

Here’s the thing about neural networks: they’re incredibly difficult and time-consuming to create and alter. The amazing thing about AutoML is that using a neural network to create other neural networks means that human programmers can delegate the heavy lifting to the AI, which is very adept at trawling through millions of nodes and collecting/changing basic information. With that kind of automation, all the programmers need to do is give it feedback on whether the networks it’s creating are doing a good job.

Here’s an example of what an AutoML-created daughter neural network looks like (right), compared to a human-designed neural network (left) meant to solve the same problem):

With this in mind, I started thinking about how a mage might use the structure of a neural network (and the techniques of AutoML) to create a doppleganger that is not only realistic and responsive, but is (for the most part) autonomous.

Now, just like a real neural network, this magical, semi-autonomous doppelganger would be a dumb automaton–maybe Turing complete, but not capable of doing anything it wasn’t instructed to do. This, however, is where deep learning comes in–the ability for neural networks to independently develop more complex layers to deal with problems. Given enough data and power and a competent neural network, there can be an element of emergence–the arising of a large phenomenon from smaller interactions.

It’s important to realize that neural networks are based off the structure of the human brain, and that when you create a new one, you’re essentially creating the possibility of a new brain to develop, one that can learn, make decisions, and change itself based on inputs. The problem, however, is allowing the system to change itself–as XKCD brings up, you could make a fully functional computer with rocks and enough space, but it would be extremely slow. So how could a neural network-like spell develop and change itself?

What I imagine is a mage who turns their body into a living canvas, with their skin becoming the hardware and the spells becoming the software. After laying down the basic structure of the neural network and employing the techniques similar to AutoML, the spell would begin to spit out output spells, which the mage would then look at and give feedback on. In this case, magic would be the stand-in for electricity, and the human body would take the place of a terminal or OS. Once the networks became complex and developed enough, the mage would essentially be walking around with a second brain on their body, operating in real time and generating a doppelganger like a projector. Creating illusions is just one use–reprogrammed, this same structure could be used for all kinds of magical purposes, including creating new custom spells.

Of course, the process of training the magical neural network and doing backpropagation would still take time, effort, and expertise, but the great thing about the AutoML system is that it can conceivably be used by non-experts to create an intermediary network that can do the more complicated tasks of creating and altering new, purpose-crafted networks. It essentially offers a shortcut to more complex creations.

In the end, it all comes back to Lain and Ghost in the Shell–can we create a facsimile of a person with the emergent property of consciousness? At what point does the illusion become indistinguishable from reality? When do we give up on our senses to tell us who is real and who isn’t? Who slips into my robot body and whispers to my ghost?

Hidden Layers: Spell Maps, Illusions, and Neural Networks

Meow Wolf and the House of Eternal Return

This past week, I visited my old college roommate in Albuquerque and went on a road trip to Santa Fe to check out Meow Wolf, which is home to something called ‘The House of Eternal Return.’ The building contains a full-sized family house, complete with a living room, porch, kitchen, and bedrooms, but scattered around the house are books, planners, and pamphlets that give clues about the residents, including Lucius Fox, who is the founder of a Scientology-like cult called Positive Mechanics, which is concerned with travelling through dimensions. It’s essentially Mark Danielewski’s House of Leaves, but built to scale, with a refrigerator that leads…somewhere else.

While wandering around the house, I found some interesting in-world texts, including a will, a PowerPoint presentation on a computer about Positive Mechanics, and a cipher for an unknown language, which allows you to decode a nearby message in a picture frame. Most interesting of all was a tapestry titled the ‘Technomancer Manifesto,’ which can be heard spoken aloud here.

Here are some of the pictures:


The House of Eternal Return is hands-down the coolest place I’ve ever visited, and not just because it’s a funhouse filled with occult B.S. It’s the kind of weird, incredibly ambitious project that you always hear people talk about as some ultimate goal, but inevitably never gets off the ground due to practicality. But Meow Wolf and the HOER is not only real, it exceeds all expectations and all goddamn definition. It really is a playground for the imagination, and mixes dark storytelling with mind-bending experiences and the sheer joy of exploring–anyone who’s been there knows the stuff I’ve described is only one-tenth of the experience. It makes me happy that a place like HOER exists, and it inspires me to do something just as weird and ambitious.

You can check out the website here.


Meow Wolf and the House of Eternal Return