July 20, 2019

Reality Is Like Pokemon Go

“Released in July 2016, Pokémon Go is a location-based, augmented-reality game for mobile devices, typically played on mobile phones; players use the device’s GPS and camera to capture, battle, and train virtual creatures (“Pokémon”) who appear on the screen as if they were in the same real-world location as the player: As players travel the real world, their avatar moves along the game’s map. Different Pokémon species reside in different areas—for example, water-type Pokémon are generally found near water. When a player encounters a Pokémon, AR (Augmented Reality) mode uses the camera and gyroscope on the player’s mobile device to display an image of a Pokémon as though it were in the real world.* This AR mode is what makes Pokémon Go different from other PC games: Instead of taking us out of the real world and drawing us into the artificial virtual space, it combines the two; we look at reality and interact with it through the fantasy frame of the digital screen, and this intermediary frame supplements reality with virtual elements which sustain our desire to participate in the game, push us to look for them in a reality which, without this frame, would leave us indifferent. Sound familiar? Of course it does. What the technology of Pokémon Go externalizes is simply the basic mechanism of ideology—at its most basic, ideology is the primordial version of “augmented reality.”

The first step in this direction of technology imitating ideology was taken a couple of years ago by Pranav Mistry, a member of the Fluid Interfaces Group at the Massachusetts Institute of Technology Media Lab, who developed a wearable “gestural interface” called “SixthSense.”** The hardware—a small webcam that dangles from one’s neck, a pocket projector, and a mirror, all connected wirelessly to a smartphone in one’s pocket—forms a wearable mobile device. The user begins by handling objects and making gestures; the camera recognizes and tracks the user’s hand gestures and the physical objects using computer vision-based techniques. The software processes the video stream data, reading it as a series of instructions, and retrieves the appropriate information (texts, images, etc.) from the Internet; the device then projects this information onto any physical surface available—all surfaces, walls, and physical objects around the wearer can serve as interfaces. Here are some examples of how it works: In a bookstore, I pick up a book and hold it in front of me; immediately, I see projected onto the book’s cover its reviews and ratings. I can navigate a map displayed on a nearby surface, zoom in, zoom out, or pan across, using intuitive hand movements. I make a sign of @ with my fingers and a virtual PC screen with my email account is projected onto any surface in front of me; I can then write messages by typing on a virtual keyboard. And one could go much further here—just think how such a device could transform sexual interaction. (It suffices to concoct, along these lines, a sexist male dream: Just look at a woman, make the appropriate gesture, and the device will project a description of her relevant characteristics—divorced, easy to seduce, likes jazz and Dostoyevsky, good at fellatio, etc., etc.) In this way, the entire world becomes a “multi-touch surface,” while the whole Internet is constantly mobilized to supply additional data allowing me to orient myself.”

Read more

JJ Editor's Daily Picks

"The likely successor to British Prime Minister Theresa May, Boris Johnson, has plans to subsume the government department overseeing development aid into the foreign office, effectively eliminating it. That will destroy a post-Brexit United..."

"Gerard Baker, editor-at-large at the Wall Street Journal (no reflexively anti-Trump publication) recently wrote a piece decrying Donald Trump and his foreign policy as a fount of erratic unpredictability. This essay will give the counter view...."

"On Wednesday, Congresswoman Ilhan Omar announced that she will be visiting Israel and the Palestinian territories in the coming weeks. Omar will be accompanied by Rep. Rashida Tlaib. The two freshman congresswomen have become a focal point of..."

"Netflix may have lost US subscribers for the first time since it began making its own shows, but that didn't stop the streaming giant from dropping new figures about how many people are sucked into its Adam Sandler vortex. (Spoiler: More than..."

"A few years ago, Amy Balliett, CEO of a Seattle-based design and marketing firm, noticed that as the work week slogged on, her employees’ energy and productivity wilted. “That would slump to such an extent that the same task on Monday would..."

"Over the last few days the #faceappchallenge has taken over social media. This “challenge” involves downloading a selfie-editing tool called FaceApp and using one of its filters to digitally age your face. You then post the photo of your wizened..."

"Although there are plenty of irrational aspects to life in modern America, few rival the odd fixation on lawns. Fertilizing, mowing, watering — these are all-American activities that, on their face, seem reasonable enough. But to spend hundreds..."

"Can a book change the way we think? I don’t mean that in the sense of a reader’s opinion or ideology shifting—of course the right literary work can do that. But can a book rewire the brain itself, literally changing the way one particular mind..."

"It’s our job to let kids know we see and hear them, but we’re not necessarily going to solve siblings’ conflicts for them (or else they never get the practice). When squabbles start, imagine you’re a sportscaster and describe what you see in..."

"Magali Trejo-Martinez, a 22-year-old living in Salem, Oregon, recently went on a date that was rather uninspiring. “I had dinner, had a couple margaritas, and then went home,” is how she recapped the evening. This outcome wasn’t entirely..."

"The first lunar landing was many things — a D-Day-like feat of planning and logistics, a testament to the power of man's will, an ostensible propaganda coup for NATO. It was also, I think, one of the most misunderstood events in the history of..."

"THE FIRST TIME Bernie Sanders ran for president, he didn’t talk much about being Jewish. In fact, he didn’t talk much about himself at all. His 2016 primary campaign, like his whole political career, was relentlessly focused on one topic: income..."