Facebook recently purchased a facial recognition software company that promises to “bring more fun effects to photos and videos” posted on the social media behemoth. The origins of the technology — named “FacioMetrics” — is much more sinister, however, and is likely to be used in much less lighthearted ways.
In its announcement of the acquisition, Facebook highlights FacioMetrics’ ability to “allow people to express themselves in fun and creative ways” and to “build even more engaging sharing experiences on Facebook.” There isn’t, however, any mention of how, and more importantly why, FacioMetrics was created.
To answer that question, we must follow a few steps that leads from a technology developed by the Carnegie Mellon Institute for Robotics to the “fun and creative” technology now part of Facebook, a company that boasts more than 1.8 billion users.
A press release issued by Carnegie Mellon University on December 16, 2015 describes the story of a recently developed “image analysis software.” The story sheds new light on just how powerful this so-called photo filter really is and just how invasive its possible applications are. Here’s the background on FacioMetrics, back when it was called IntraFace:
Automated facial analysis is at the heart of a host of potential applications, from monitoring the emotional state of patients to detecting whether a public speaker is losing an audience’s attention. Fernando De la Torre, associate research professor in the Robotics Institute, said releasing the latest version of the software, called IntraFace, will help expand those applications by giving researchers access to its state-of-the-art capabilities.
“IntraFace provides a breakthrough in facial feature tracking that simplifies the problem of facial image analysis, working rapidly, accurately and with such efficiency that it can run on most smartphones,” De la Torre said. “Now it’s time to develop new applications for this technology. We have a few of our own, but we believe there are lots of people who may have even better ideas once they get their hands on it.”
In May, a story from NPR revealed what one of those “better ideas” might be, especially when it comes from the fertile minds at Facebook.
Every time one of its 1.65 billion users uploads a photo to Facebook and tags someone, that person is helping the facial recognition algorithm. The tag shows the algorithm what someone looks like from different angles and in different lights, Frankle says. If you give Facebook a face to identify, it has fewer photos to parse through, because it’s only looking at photos of you and your friends.
Facebook, according to the company, is able to accurately identify a person 98 percent of the time. Compare that with the FBI’s facial recognition technology, Next Generation Identification, which according to the FBI, identifies the correct person in the list of the top 50 people only 85 percent of the time.
One need only imagine how much more accurate Facebook’s facial recognition algorithms will be now that FacioMetrics is on campus at Menlo Park.
The prospects of how invasive these “better ideas” could be should frighten the 1.3 billion active monthly users of the site.
Particularly as Facebook isn’t known for more than token resistance to cooperating with the federal government’s quest to put everyone under the National Security Agency’s (NSA) never-blinking eye.
According to a statement posted on the company’s website in June 2013, government agencies — including federal, state, and local authorities — requested user data on between 18,000 and 19,000 account holders.
Following the negotiations in 2013 that opened the way for Facebook to report its cooperation with requests to hand over user information, Microsoft made a similar surveillance disclosure. A blog post on the Redmond, Washington-based company’s website declared:
For the six months ended December 31, 2012, Microsoft received between 6,000 and 7,000 criminal and national security warrants, subpoenas and orders affecting between 31,000 and 32,000 consumer accounts from U.S. governmental entities (including local, state and federal).
Altogether, that means the accounts of approximately 50,000 Americans — accounts they believed were secure — were laid open to the eyes of government agents.
These revelations may be nothing more than cover fire to distract users from the collusion of these corporations with the NSA as disclosed by NSA whistleblower and former NSA subcontractor Edward Snowden.
Under the PRISM data-gathering program, the NSA and the FBI are “tapping directly into the central servers of nine leading U.S. Internet companies, extracting audio, video, photographs, e-mails, documents and connection logs that enable analysts to track a person’s movements and contacts over time,” as reported by the Washington Post.
The joint venture has been functioning since 2007, but came to light only in a PowerPoint presentation that was part of the cache of documents leaked by Snowden.
Snowden claimed that the program was so invasive that “They [the NSA and the FBI] quite literally can watch your ideas form as you type.”
According to the information Snowden released, two of the tech companies that disclosed government surveillance requests — Facebook and Microsoft — were giving the government access to the private information of millions of users.
They were not alone, however. Yahoo, Google, PalTalk, AOL, Skype, YouTube, and Apple all allowed the agents of the federal surveillance state to secretly snoop on their users.
The New American has reported on the story in detail:
PRISM works in conjunction with another top-secret program, called BLARNEY, which, according to the program’s summary, “leverages IC [intelligence community] and commercial partnerships to gain access and exploit foreign intelligence obtained from global networks.”
PRISM allows the NSA to enter a company’s data stream and extract communications by keying in “selectors” or search items. The agency is mandated by law to conduct surveillance only on foreign operations within the United States, but the selectors are designed to produce at least 51 percent confidence in the “foreignness” of the data it collects, meaning it could be intercepting wholly domestic communications nearly half of the time. Training materials instruct new analysts to submit accidentally collected U.S. content for a quarterly report. But the training instructions also tell the analysts that “it’s nothing to worry about,” the [Washington] Post said.
Recognizing a person’s face, though, is not nearly the limits of the complex computations FacioMetrics can reach.
Beyond simply identifying the user from photos or videos posted by the one billion members of Facebook who use the company’s mobile app every day, the FacioMetrics technology is designed to detect emotions and attitudes. Again, here’s the story from the 2015 Carnegie Mellon article:
To increase its efficiency and help it work reliably with most faces, the researchers used machine learning techniques to train the software to recognize and track facial features. The researchers then created an algorithm that can take this generalized understanding of the face and personalize it for an individual, enabling expression analysis.
The result is that IntraFace is both accurate and fast. It occupies less computer memory than other methods and requires less power to run, making it suitable for use on a wide range of platforms, including smartphones and embedded systems.The potential applications are many, including distracted or drowsy driver detection, automated analysis of marketing focus groups, animation of avatars in multi-player video games, human-robot interaction, and monitoring or detection of depression, anxiety and other disorders.
In light of the federal government’s effort to prohibit veterans seeking help for Post-Traumatic Stress Disorder (PTSD) and other mental illnesses from purchasing or owning firearms, one need only connect the dots from IntraFace to FacioMetrics to Facebook to PRISM to the NSA to realize that the forces of federal surveillance just got a whole lot more mobile and a whole lot more intrusive.
