Facebook Wants To Listen In On What You’re Doing

Kashmir Hill

Forbes Staff

Facebook had two big announcements this week that show the company’s wildly divergent takes on the nature of privacy. One announcement is that the company is encouraging new users to initially share only with their “friends” rather than with the general public, the previous default. And for existing users, the company plans to break out the old “privacy dinosaur” to do a “ check-up” to remind people of how they’re sharing. Facebook employees say that using an extinct creature as a symbol for privacy isn’t subtle messaging, but simply an icon to which their users respond well. Meanwhile, Facebook’s second announcement indicated just how comfortable they think their users are in sharing every little thing happening in their lives. Facebook is rolling outa new feature for its smartphone app that can turn on users’ microphones and listen to what’s happening around them to identify songs playing or television being watched. The pay-off for users in allowing Facebook to eavesdrop is that the social giant will be able to add a little tag to their status update that says they’re watching an episode of Games of Thrones as they sound off on their happiness (or despair) about the rise in background sex on TV these days.

Facebook's animal of choice to represent privacy is an extinct one

“The aim was to remove every last bit of friction from the way we reference bits of pop culture on the social network,” writes Ryan Tate of Wired. Depending on how you feel about informational privacy and/or your friends’ taste in pop culture, that statement is either exhilarating or terrifying.

The feature is an optional one, something the company emphasizes in its announcement. The tech giant does seem well-aware that in these days of Snowden surveillance revelations, people might not be too keen for Facebook to take control of their smartphone’s mic and start listening in on them by default. It’s only rolling out the feature in the U.S. and a product PR person emphasized repeatedly that no recording is being stored, only “code.” “We’re not recording audio or sound and sending it to Facebook or its servers,” says Facebook spokesperson Momo Zhou. “We turn the audio it hears into a code — code that is not reversible into audio — and then we match it against a database of code.”


If a Facebooker opts in, the feature is only activated when he or she is composing an update. When the smartphone’s listening in — something it can only do through the iOS and Android apps, not through Facebook on a browser — tiny blue bars will appear to announce the mic has been activated. Facebook says the microphone will not otherwise be collecting data. When it’s listening, it tells you it is “matching,” rather than how I might put it, “eavesdropping on your entertainment of choice.”

It reminds me of GPS-tagging an update, but with cultural context rather than location deets. While you decide whether to add the match to a given Facebook update, Facebook gets information about what you were listening to or watching regardless, though it won’t be associated with your profile. “If you don’t choose to post and the feature detects a match, we don’t store match information except in an anonymized form that is not associated with you,” says Zhou. Depending on how many people turn the feature on, it will be a nice store of information about what Facebook users are watching and listening to, even in anonymized form.

Sure, we’re used to features like this thanks to existing apps that will recognize a song for us. But usually when you activate those apps, you’re explicitly doing so to find out the name of a song. Facebook is hoping to make that process a background activity to composing a status update — a frictionless share that just happens, the real-world version of linking your Spotify account to your social media account allowing playlists to leak through. Facebook spent a yearhoning its audio sampling and developing a catalog of content — millions of songs and 160 television stations — to match against. It’s obvious that it wants to displace Twitter TWTR +10.69% as the go-to place for real-time commenting on sporting events, awards shows, and other communal television watching. “With TV shows, we’ll actually know the exact season and episode number you’re watching,” says Zhou. “We built that to prevent spoilers.”

So the question now is whether people are willing to give Facebook eavesdropping powers in exchange for a little Shazam.



Google Chrome Can Eavesdrop On Your PC Microphone

Marc Weber Tobias

Marc Weber Tobias, Contributor

I am an investigative attorney and physical security specialist.


What if the laptop on your desk is listening to everything that is being said during your telephone calls and conversation or from others near your computer? Then imagine that the audio from the internal microphone is being instantly uploaded to Google where it is transcribed and broadcast on a real-time basis to a malicious web site, Twitter, or to a competitor. Sound like a high-tech novel?

This scenario is not only possible (and easily accomplished) but I had a researcher in Israel do this last Saturday with my laptop to confirm the information that appeared in the New York Times and many other publications last week. Anyone that uses voice recognition built into Google Chrome browsers (and soon others) should pay attention because of the potential for eavesdropping and interception of conversations within several feet of any computer running this browser.

Tal Ater is a voice recognition specialist living near Tel Aviv. In September, 2013 he notified Google about a bug he discovered in Chrome that could allow your computer to act as an ‘open microphone” and send the digital audio to Google for them to process through their highly efficient speech recognition software.

This is done on a real-time basis and Google returns the text translation back to their Chrome browser for use by whatever website it is logged into. The problem that Tal Ater found was that you could leave the site but audio processing could under certain conditions continue to occur unless you closed the pop-up window that was present during the session.

Before Ater notified Google engineers, the pop-up that showed microphone status was in the background and could easily be missed by a user. Since the notification, they changed that so that the window is now in the foreground. But it really does not matter unless you are paying attention and understand the vulnerability. Once you give permission to open your microphone it may stay connected and the permission can remain active.

I met with Mr. Ater in Tel Aviv to discuss his research and the real–world threat, if any, that was posed by his findings. Watch my interview to more fully understand the way that Google configured Chrome and what every computer user should do to protect against this kind of possible electronic intrusion.

Google Chrome is one of the most popular and sophisticated browsers and is the only one that presently features the capability to talk to it in order to generate commands and search data that is sent to web sites. Google is so good at voice recognition that they can, upon request, send multiple iterations of speech-to-text conversations if the user questions the accuracy of the translation or the syntax. That means an eavesdropper has a better chance to improve the accuracy of the speech-to-text processing.

I asked Mr. Ater to set up a phony stock quote website that would accept voice search commands for different stocks. A clever idea for busy traders: just speak the name of the stock and their computer will display price and trading data. The site was named Stock Talk.

When I logged into Stock Talk with my Chrome browser, it asked for permission to use my microphone, which I gave. Then I could talk to the site. I logged out of that site and went to others, all the while talking on Skype to Mr. Ater, and also walking around my hotel room, up to about eight feet from my laptop. Within a few seconds, Tal reported that he had received a fairly accurate text translation of both my speech and his when we were conversing. Then he uploaded each snippet, in real-time, to Twitter. This is the modern version of the scene in the movie MASH in which they broadcast an sex scene throughout an army base over the loudspeaker.

The modern version of that scene could allow confidential discussions to be transmitted in text to anyone in the world on an almost-immediate basis. One night later, the audio permission was still enabled on my computer which means if I logged into a malicious site it would capture text of any audio near my computer.

Here is what you need to know about this vulnerability:

  • Such an exploit is dependent on the user having approved the use of the microphone on a vulnerable site. Every site asks for permission and if the user never approves use of the microphone on the vulnerable site, there is no exploit;
  • Neither Google nor any malicious website captures the actual audio from your computer. The digitized file is uploaded and processed, and the text version is returned to the browser;
  • If you log into a malicious site it can capture text of the audio of anyone near your computer;
  • The permission to turn on your microphone will remain active until cancelled, which means a site can continue to monitor your speech without your knowledge;
  • You can view the sites you have visited and allowed the use of your microphone in the Chrome browser, under advanced settings (at bottom of the page), then Privacy-Content Settings-Media-Microphone;
  • Google cannot distinguish who is speaking, as with traditional audio recordings, especially if there are several different audio sources present;
  • If a computer is configured through a malicious site to capture audio and distribute the text to third parties, it may constitute a violation of state or federal law relating to unauthorized intercept, use, or disclosure;
  • Audio may be uploaded, even if you are not using your computer. See the demonstration that Tal Ater first released;
  • Any window that has been enabled by the user to turn on the microphone will show an indicator that the microphone is on;
  • The speech recognition feature in Chrome is designed to ensure users are in control, and that the use of microphone is transparent. Users must enable speech recognition for each site that requests it.
  • The current version of Chrome does not allow for hidden popups that stay in the background without the user’s knowledge. The window would pop-up in front of other windows.