RESISTANCE IS FUTILE

WE LET TODAY'S ALGORITHMS MAKE MORE DECISIONS FOR US THAN WE DO OURSELVES. WILL THE CYBERSERVANT SOON BECOME THE MASTER? H Humankind still rules the world we live in, but the seeds of our demise were, like ancient DNA, planted long ago. Fate and free will, always fodder for philosophical sparring, have become the subjects of even murkier absolutes as algorithms—snips of computer code—insert themselves into nearly ev­ery decision and process in our lives. Even the most complex bits of life can be reduced to binary decisions: If this, then that; if that, then this. It's in this environment that the algorithm thrives. And it's in this environ­ment that we live. Thanks to ever- increasing computing speeds and miles of code accounting for nearly every eventuality, algo­rithms have quickly risen to assume control of many facets of our lives, from Google search to Spotify, from insurance rates to which route we drive on the way home from work. Algorithms can even read our minds. It's that last development that holds either the most prom­ise or the most peril, depending on with whom you speak. It's difficult to identify a seminal mo­ment when algorithms' march toward dominance tipped toward critical mass. For years, the takeover advanced silent­ly. Only when its effects became indel­ible, when algorithms' rise in society be­came stark, palpable, did people sound the alarm. Needless to say, humans are outmatched. Where we see chaos and unfathomable amounts of data, bots driven by algorithms detect patterns, discern order and make conclusions. In a way, algorithms and the software that employs them compose a fold of human evolution. Every day, Silicon Valley proposes to outsource more of our lives' mundane activities. It's a seductive proposition. Why worry about driving when BMW, Audi and others make algorithms for that? Why waste glove- compartment space on a map when our phones, as directed by Apple or Google, can direct us where to go? Why bother working the bar crowd when a dating-site algorithm from OkCupid can deliver matched per­sonalities by the dozens? Why write down shopping lists when we can bark at our refrigerator "More milk!" and have it delivered by Amazon in an hour? We have designed algorithms to decode our behavior and our brains, and they have succeeded. The only question: Now what? With any luck, the answer hu­mans wish for will be in agreement with that of the bots. Thirty years ago algorithms first gained notoriety by cracking our financial markets and giving tech-sawy firms the edge on trading floors. Since then they've moved on to affect, if not yet control, every aspect of our lives: They decide what jobs we work, whom we marry, where we live, where we drive, what prescriptions we receive, what music we hear, what grades we get and how our money is invested. Algo­rithms have invaded those nuanced bas­tions where it would seem impossible to replicate a human's understanding and touch, tasks such as creating original music, grading written essays, writing original fiction and playing games like poker that mix logical processes with nonlinear takes on human emotions. So what are these things, these algo­rithms that are so well poised to replace us? While the name carries a whiff of technical erudition, an algorithm is a simple device. It's a set of instructions that, given input, produces output. An algorithm needn't involve computers. A set of instructions for making morn­ing coffee by hand is technically an algorithm. Of course, algorithms can also involve thousands of inputs, data­base queries, calculations and dynamic, evolving computations. One of the first algorithms many engineering students are required to compose in basic com­puter science courses is one that will play a perfect game of tic-tac-toe. The inputs are the moves of the human; the outputs, the moves of the computer. All computer languages—C, Java, Ruby, Python, PHP, whatever—are vehicles created to express algorithms. These days, the powers of prediction residing in computer code make tic-tac-toe pro­ grams analogous to the sticks chimpan­zees use to harvest termites. • The specter of somebody, something, being able to read our thoughts and our intentions by parsing our words seems incredible. But it doesn't seem impossible. We assume that psycholo­gists operate in something of a similar fashion, though their feedback is nei­ther as demonstrative nor as prompt as that of algorithms. In all these cases, we give algorithms and therapists a lot to work with: We answer questions, we make statements, we talk and talk until the words pile up into the hundreds or thousands. But what if we gave them nothing? What if we offered no words, no typing, no hand gestures—-just our faces? Could algorithms still read us? My three children, like most, have an affinity for television. At home we limit the time they're allowed to watch, but given the chance, they will turn their attention wholly over to the pixels in front of them. Outside noises, such as that of a parent asking a question, are rendered nonexistent. Their faces, it seems to me, settle into a kind of open- jawed stupor that can stay frozen for the entire length of the program they're watching. What happens on the screen effects no change on their Little coun­tenances. Or at least that's my view of things, the human view. Algorithms, however, can sniff out our brains' inner workings during times like this, when we're offering few palpable clues. Even seemingly vacant expressions, like those on children watching television, offer data that can be parsed by tools that are sensitive enough to detect them—tools wielded by algorithms, of course. Several companies are developing this kind of technology, using algo­rithms to read people's faces as portals to their brains. One of the companies, Emotient, has a direct lineage to Paul Ekman, who 60 years ago be- gan to study the meaning of facial expressions. Ekman linked dif- ferent movements of the lips, brow, cheeks and forehead to six distinct emotions: hap­piness, sadness, anger, dis- gust, fear and surprise. After spending more than 20 years on the subject, Ekman in 1978 published what he called FACS, or Facial Action Cod- ing System, which categorizes every facial expression. FACS provides a set of standards to decode every natural facial move­ment, from a slight upturn of the lips to a nose crinkle and an eyebrow dip. After studying the human face and all the ways emotion distorts it, Ekman had classified each derivative of every expression imaginable. A system so comprehensive has to be complicated. For human psychologists or anybody else, it can take years to master. It's why Ekman himself, now 81, has for 30 years been one of the most sought-after consultants in the world. He has done work for the CIA, the FBI, DreamWorks and Pixar, teaching people at these places the taxonomy of facial movements. When Ekman was developing FACS in the 1970s, it oc­curred to him that this kind of analysis could one day be packaged into com­puter code, a collection of algorithms that could recognize every tiny grimace, every eyebrow tilt. "I absolutely (continued on page 116) RESISTANCE IS FUTILE Continued from page 52 thought it might be possible to automate this, but at that time the computer power just wasn't available," Ekman explains to me. In 1985, while attending a conference on his FACS system in Wales, Ekman met a sci­entist who had developed one of the world's first parallel-processing computers. One of the computer's first applications involved algorithms that recognized human faces at a distance of 50 yards. But when people presented any kind of nonneutral mien, it created enough "noise" that the algorithms no longer recognized them. The noise of people's emotional expressions consistendy foiled the facial-recognition software. "But that noise," Ekman says, "was my focus." Ekman visited the computer lab in Lon­don and experimented with the machine's power for a week, during which time he was able to program it to recognize several basic facial expressions. Following this ex­perience, Ekman wrote a grant proposal for the National Institute of Mental Health to pursue the work further. But the NIMH told him, succincdy, "We don't think com­puters can do what you think they can," remembers Ekman. Soon after that, Ekman met Terry Sejnowski, who had a Ph.D. in physics from Princeton and was a researcher and profes­sor at the University of California, San Diego. Sejnowski helped Ekman get his study funded, and the two began to work on auto­mating the task of reading human faces for everything they betray. Joining their project was doctoral student Marian Bardett, who in the 1990s began to apply machine-learning algorithms to the problem. Machine-learning algorithms can be pow­erful tools when unraveling large, compli­cated riddles for which composing enough linear programming—clear, prescriptive algorithms—would be impossible. Given a set of desired outcomes, a machine-learning algorithm will work to find the most efficient ways to reach similar outcomes with new problems. The more data such algorithms consume, the smarter they become. This is why they're effective in teasing out nonintui- tive relationships within large sets of data. Bartlett's use of machine-learning al­gorithms proved successful. Her col­leagues' and her work eventually formed the foundation of Emotient, which took its face-reading product to market in 2013, after advances in digital cameras and off-the-shelf processing power made the technology applicable to a wide audience. At this point the software has become fer better than any human at reading feces. "If you ask people to make subjective judgments on what a face is telling them, they're often wrong. People don't know what to look for," says Bardett, co-founder and lead scientist at Emotient. "But when you measure objective­ly, there is a huge amount of information." The first clients for Emotient's algorithms came from Madison Avenue, as advertis­ers wanted to pair the face-reading technol­ogy with their normal practice of using fo­cus groups to determine what kinds of new products should be released. Procter & Gamble, for one, used Emo­tient's algorithms to gauge consumer reac­tion to new detergent scents. P&G asked the people in its focus group to sample the fra­grances and then, as is standard, had them fill out a survey of their thoughts about all the product variations. At the end of the event, the participants were allowed to take home any detergent of their choice. As it turned out, the fragrance people reported as their favorite in the survey was usually not the one they chose to take home. Emotient's algorithms, however, predicted which scent a person would take home with a high degree of accuracy. P&G recorded the focus group members taking their first smell of each fragrance. Initial reactions, gut reac­tions, were betrayed by slight changes in their fecial expressions, usually lasting far less than a second, when they got the first whiff of a scent. That gut reaction, driven by the brain's amygdala, is what dictates most decisions. The amygdala is separate from the part of the brain that drives logic and speech, which are what produce the results in participant surveys. The idea that focus-group surveys are nearly worthless sent a shudder through the advertising industry, and delivered a regular stream of well-paying clients to Emo­tient ever since. Emotient's technology has advanced to the point that it can gauge and measure every face in a frame of video. A high-resolution video clip of the NBA finals, for instance, can be evaluated by Emotient's algorithms to determine the general disposi­tion of the crowd during that moment of the game. It could be 100 feces or 500 feces. The algorithms see and read them all. The technology Emotient employs has be­come so efficient, so fest, the company now of­fers access to its algorithms to anybody via the web. Users upload their videos to Emotient's site and pay $ 1.99 per minute for analysis. I felt compelled to test the algorithms on my gaping children as they watched televi­sion. It turns out even their rather stoic feces tell a big story. I recorded as they watched the beginning of The SpongeBob Movie: Sponge Out of Water and uploaded the 15-minute video to Emotient's servers for analysis. I didn't have to wait long. In a couple of hours Emo­tient's system returned a report to me, one impressive in its depth and thoroughness. Among the data provided to users, Emo­tient returns a version of the original video augmented with frames that oudine peo­ple's feces. When a person in the video looks away from the camera, the frame disap­pears. Next to each frame, the software dis­plays a single word describing the emotions of that person at that precise moment. For my little video watchers, the software's reg­istered emotion was often "neutral," or the same evaluation most people would make when seeing the empty looks produced by children absorbing on-screen entertain­ment. But at different points of the movie, cracks of feeling would flit across their faces. The same moment that scared one of my younger kids—duly noted by Emotient— instilled bemusement in the older one. The software then knit all these moments into single story lines for each child. Even a lay­man could see the inflection points of the movie and how they affected each viewer. The software isn't perfect, however. It mistook one of my daughters' emotions for "disgust" during a 10-minute period when she put her hand on her chin and left it there. But it works well enough that we should expect algorithms to one day lurk in every store camera, every political rally, every car dealership, even job interviews— anyplace where discerning the inner reac­tions of people is paramount. This reality doesn't sit well with Ekman, creator of all the logic behind the algo­rithms. He remains keenly interested in the science of his system and, as an advi­sory board member of Emotient, holds equity in the company, but he surprises me by saying, simply, "I'm quite worried." He explains further: "If you're going to analyze people's expressions and analyze their emotions, I think you should have their consent." At this point, Emotient says explicit con­sent isn't necessary because it keeps the data anonymous. Faces in a crowd are just feces in a crowd. But Ekman feels that reading somebody's emotions so mechanically, al- gorithmically, entails a violation of privacy. The questions surrounding this use of algorithms insert sci-fi plots into real life. Where do we draw the line? Where does the utility of code stop? • Nicholas Carr, author of The Glass Cage, wor­ries that automation's march has rendered us stupider, that algorithms demote humans to lever operators who let computer code do all the real work, whether in the cockpit of a plane or on the machinist bench in a factory. "Automation severs ends from means," he writes. "It makes getting what we want easier, but it distances us from the work of knowing." When we surrender, as Carr says, the work of knowing, we are capitulating to the power of bots. Carr advocates for humans to spend more time at labor without the artifi­cial proxy of software between them and the job. He points out that studies have shown that airline pilots' skills degrade when they forfeit most of their flying time to autopilot algorithms. While it's true that autopilots are one of the reasons air travel has become in­credibly safe, Carr argues that pilots should be flying by hand more often, which would keep their skills honed and help mitigate the human errors that have led to most of the major air disasters of the past two decades. "We can allow ourselves to be carried along by the technological current, wher­ever it may be taking us, or we can push against it," writes Carr. Such talk evokes thoughts of Star Trek's vil­lainous Borg, a race evolved from a mixture of man and machine whose regular advice to those it conquers, "Resistance is futile," has become a cultural refrain. Borg aside, the possible takeover of the world by algorithms infused with artificial intelligence has been discussed for decades. Hollywood has long been intrigued by this plot, from Stanley Kubrick's 2001: A Space Odyssey to the Terminator franchise and, more recently, Johnny Depp's Transcendence. Com­puter scientists, however, have dismissed such tales as hyperbolic and unlikely. But in the past year, three of the biggest minds in science have separately expressed warnings about software so intelligent it could seize humanity's place as Earth's dominant force. Bill Gates, the most suc­cessful software entrepreneur of all time, said, "I am in the camp that is concerned about super intelligence." Stephen Hawking, the brilliant theoreti­cal physicist, told the BBC, "The primitive forms of artificial intelligence we already have have proved very useful. But I think the development of full artificial intelligence could spell the end of the human race." Perhaps most foreboding are the thoughts of businessman and inventor Elon Musk, who has repeatedly sounded the alarm with tweets such as "We need to be super careful with Al. Potentially more dangerous than nukes." Musk has consistendy done things that others considered impossible: building Tesla Motors into a major force and founding SpaceX, a private company that designs, manufacturers and launches rockets into space for a fraction of the money NASA and other space-race in­cumbents have spent. His declared worry on the subject deserves attention. • Algorithms have already evaluated many of us to a degree comparable to that of a human psychologist's scrutiny. And most of us have no idea it has happened. In my book Automate This, I profile how Chicago company Matter- sight built a library of 10 million algorithms to categorize human speech. The company's engineers married these algorithms with speech recognition to create a system that determines a speaker's exact personality type and, often, what he or she is thinking. The results can be startlingly accurate. The system correctly tagged me as having what is called a thoughts-driven primary personality and a reactions-driven secondary personality. Often when we call customer-service lines we get a recorded refrain: "This call may be monitored or recorded for quality- assurance purposes." We assume this has something to do with training or liability. But it often means that millions of algo­rithms have setded in to listen to us. When the bots know our personalities, they know how to treat us to keep us happy and on­board as profitable customers. By routing our calls to operators with personalities similar to our own, the bots keep customer- service calls mercifully short—and cheap. What was something of an experiment when I first wrote about it has become a sweeping movement within consumer- facing companies. Mattersight CEO Kelly Conway recently told me his algorithms have now profiled 20 percent of American adults' personalities. The Google search algorithm, perhaps the most powerful in the world, decides much of how we go about our lives and be­comes increasingly tailored to our tastes the more we use it. It directs where we eat, what businesses we patronize, where we decide to live, travel, go to school, raise a family. Most people's web interactions begin with the Google search box. What it decides to put on the first page—or prepopulate before we even finish typing our thoughts—is pivotal, whether we're searching for good Thai food or the best ski resort for early-season snow. We needn't know a restaurant's name any­more, as Google's algorithm will figure out all the details for us. Marketers have long known that our on­line behavior reveals a great deal about who we are. The government knows this too. The National Security Agency, Edward Snowden told us, used algorithms to determine wheth­er or not someone was a U.S. citizen, as only the communications of noncitizens can be monitored without a warrant. But the algo­rithms didn't access data about birthplace or parents; they made this critical judgment based on a person's browser and web-surfing histories. Faceless computer code was, in ef­fect, the arbiter of U.S. citizenship and the right to privacy it confers. In fact, the United States is now testing algorithms in lieu of human guards and in­terrogators. Not only are they cheaper than humans, but they're better at patrolling our borders. Through kiosks installed at border crossings, the algorithms quiz travelers and analyze their answers, examining word choice and looking for vagueness, pauses and other signs of lying. The algorithms also ingest data from high-definition cameras that measure travelers' facial expressions and eye move­ments. So far, the bot has proved far more ef­fective than humans at finding liars. In a test of the technology at a Polish border crossing, the algorithms were effective 94 percent of the time in sussing out test participants who tried to get past the checkpoint with false an­swers and papers. Human guards who ques­tioned the same people caught none of them. The existence of the Transportation Secu­rity Administration in its current form was recendy called into question when its agents failed 67 out of 70 tests in which workers from the Department of Homeland Security tried to smuggle feke explosives, weapons and other contraband past airport check­points. The TSA, already a fevored target of commentators on the left and right, has never been less popular. Some, only half in jest, have suggested replacing agents with bomb-sniffing dogs. Ekman, the wizard of facial expressions, thinks the TSA is just looking for the wrong thing. "We should be seeking out the bomber, not the bombs," he says. Algorithms could certainly be pro­grammed to look for fecial giveaways that in­dicate a person is hiding something or is on the brink of committing a violent act. They could also be employed at banks to alert guards when somebody wearing the wrong expression comes in the door. That algorithms could best humans at jobs seemingly essential to maintaining a civil soci­ety is unsettling to many. But it's a real trend. The state of Missouri, searching for ways to maintain consistent sentencing and reduce the $680 million burden of housing 30,000 inmates in state prisons, in 2005 implemented what's called the Missouri Automated Sen­tencing Application. Judges, prosecutors, de­fense attorneys and even magazine reporters can provide all kinds of inputs regarding the defendant, and the algorithm will provide data on sentences given to similar criminals in the past, along with information on the cost to the state of different sentences. A charge of first-degree assault for a previous offender age 22 to 34 with a high school education and full-time employment produces an average sentence of 9.3 years. The system also reveals that 7.7 percent of offenders in similar cases were sentenced to probation, 11.5 percent to some kind of treatment program and 80.8 percent to prison. The costs are included, from $9,050 for five years of probation to $167,510 for 85 percent of time served incarcerated. The Missouri algorithm used to go even further, actually providing judges with rec­ommended sentences. Although the algo­rithm's sentences were nonbinding, their existence upset enough people that Missouri legislators imposed restrictions on the sys­tem, requiring that the recommended sen­tences be removed from its output. "It's a shame, because I think the more knowledge you get to people, the better their decisions will be," says Gary Oxen- handler, a Missouri circuit court judge and the acting chair of the state's sentencing ad­visory committee. "The system is there to help you make decisions. It's a tool." Oxenhandler thinks letting algorithms into the courtroom, as long as they're not given final say, benefits the legal process. Anything that lightens his load as a judge, he says, can make him more effective in sentencing the 350 to 600 felons he may be overseeing at any one time. Scott Greenfield, a prominent New York defense lawyer whose blog has become one of the most-read legal sites on the web, finds the whole concept misguided. "Consistency here is a bit of a fool's errand," he tells me. "You can't take into account the myriad differences between human beings" that should affect their sentencing. Only humans, Greenfield insists, can apply the required nuance. The cost of prisons has become crippling for many states, including California, which released 2,700 inmates this year as part of a measure to trim spending and overcrowding. Oxenhandler thinks applying algorithms to the issue could help all states better figure out who should stay in prison and who is worth the gamble, given the savings, of being re­leased. The time for algorithms, he stresses, is now. "As the economy gets better, people aren't going to give a damn anymore," he says. "If we miss our window here, they're going to end up building more new prisons." In a job as important as this—deciding who is free and who is locked up—surely algorithms require some form of supervi­sion. Humans, however, may not be best for that job. Leading computer scientists have, more and more often, looked to algorithms to police themselves. • The algorithm the U.S. State Department uses as part of its Diversity Immigrant Visa Program is supposed to pick a group of ap­plicants at random to be awarded visas each year. In 2011, the lottery's algorithm did not work as intended but simply awarded visas to the people who had applied earliest, in order. The visas were eventually revoked and the system was rerun. The episode devastated many people who lost what they believed to be legitimate entry to the United States. Bad algorithms, bad code, the theory goes, can be prevented from doing damage when patrolled by algorithms designed for the job. It sounds ridiculous, but the concept is a rudi­mentary one within computer science. Most programmers, when creating web and mo­bile applications, create a parallel set of tests. The tests are, in effect, algorithms that patrol newly written code for ways in which it might break the application. More complex versions of these are known as accountable algorithms. Computer scientist Joshua Kroll has been pestering the State Department about its visa lottery algorithm for more than a year. The government hasn't been forthcoming about its methods, forcing Kroll to issue a formal re­quest under the Freedom of Information Act. "They could just be using a big Excel spread­sheet," he says of the State Department. "We don't really know what they're doing." Kroll would like to fix problems like this with accountable algorithms that ensure other algorithms do their jobs correctly. He thinks accountable algorithms could help solve thorny problems such as discrimination in job and credit markets, where things such as race and gender may be officially left out of consid­eration but are often inferred through indi­rect methods. Fairness is ultimately better de­termined by code than by humans, Kroll says. A world with algorithms watching our faces, measuring our words, determining who goes to jail, who gets frisked at the airport—most of that world has already ar­rived or is coming. Rather than be alarmed, some of the best-informed minds on the subject welcome algorithmic rule. David Cope is rare in that he's renowned as an artist and a programmer. He has writ­ten reams of code in Lisp, a complicated computer language favored by developers in the Al community, while also compos­ing operas and symphonies that have been performed by elite orchestras around the world. A leader in the creation of Al pro­grams that compose original music, Cope has watched classical music aficionados mis­take some of his algorithm's compositions for the work of Johann Sebastian Bach. Whereas the author Nicholas Carr ar­gues for more piloting of planes by hu­mans, Cope thinks the better answer, the obvious answer, is to get rid of humans in the cockpit altogether. Even Cope is surprised at how quickly algorithms have marched toward mastery of society. When I asked him three years ago if he thought algorithms could ever compose an original novel, his reply was curt: "No." When I asked him again this year, he'd changed his mind. He's currendy working on that very thing at his home in Santa Cruz, California. "It's natural for humans to both fear and find disgusting matters in which a machine can do better than or replace them," says Cope. "We're insecure when it comes to that. When machines can play chess or cre­ate something better, it's damned madden­ing. But I think we're gaining something. We can think on a higher level. We can now have these people who have been displaced doing something more interesting." More interesting than driving a car, for in­stance, is designing the software that drives it for us. And as Google, Audi, BMW and sev­eral other companies' self-driving autos have shown, machines have already surpassed hu­mans in this capacity. But not everybody is a software engineer. Cope concedes that some people may lose their jobs to algorithms three or four times over their careers. The key, he says, will be retraining oneself at a higher level to use the newest technology. "It seems to me that on every level when­ever we can get a machine to do a job," Cope says, "we should do it." We simply must hope that by the time algo­rithms are doing everything that they enjoy having their original creators—the soft, cor­poreal, needy and primitive versions of wet- ware called humans—around for company. ?