What if you could turn a dial and tune out an abusive tweet in your Twitter mentions or an insulting comment on Facebook?
Today, Google’s sister company Jigsaw released an experimental Chrome browser extension called Tune, which lets users control how much “toxicity” they want to see in comments across the internet. Tune is built on Perspective, Jigsaw’s API that trains machine learning (ML) models to identify and score comments that could be perceived as abusive or harassment.
At launch, the browser extension works across social media sites including Facebook, Twitter, YouTube, Reddit, and comment platform Disqus. It acts as a knob that lets users set the “volume” of conversations. Turn it up to see everything, or turn it down all the way to “zen mode” to replace toxic comments with small colored dots.
When I spoke to Perspective product manager CJ Adams for PCMag’s exclusive look inside Jigsaw’s efforts to combat online abuse and harassment, Tune was still in development. But Adams gave me some insight into how Jigsaw was thinking about the Chrome extension extending Perspective’s ML modeling to users.
“It’s about letting viewers who don’t want to sift through 10,000 comments or posts or tweets skip the toxicity by turning this dial down, or turn it up to see it all. We want to give them that control,” said Adams.
Until this point, Perspective’s technology has been tested and implemented largely as an assistive AI tool to help content moderators—at sites including The New York Times, Wikipedia, The Guardian, and The Economist—wade through the daily deluge of comments. Perspective can customize ML models to a site’s community guidelines to help filter out the trolls.
Jigsaw also worked on a pilot program with Reddit’s r/ChangeMyView subreddit, and has since helped the sub’s founders spin out their own website and more deeply integrate the Perspective API. It’s currently in private alpha testing and will soon launch a public beta.
The other use case is leveraging Perspective as a proactive tool for authors to improve conversations. The API can scan comments before they’re posted, popping up a message that might read “Before you post this, be sure to remember our community guidelines” or “The language in this comment may violate our community guidelines. Our moderation team will review it shortly.” Perspective’s toxicity scores are an indication not of severity, but probability: higher scores represent a higher likelihood that patterns in the text resemble patterns in comments people have tagged as toxic.
Neither of those applications of the technology address the places rife with the worst online abuse and harassment: social media.
Rather than work with Facebook, Twitter, and YouTube directly on another form of AI-assisted content moderation, Jigsaw wanted to give users that control. Social platforms already have countless algorithms running to flag and filter inappropriate content, but time and again they’ve proved woefully inadequate. Human moderation comes with a steep emotional cost as well, as The Verge chronicled in its recent exposé on the lives of third-party Facebook content moderators.
Tune is a test to see what happens when you take part of the onus off the platforms and let users decide for themselves, curating what they see based on their own preferences and perceptions.
The first experimental version of the Chrome extension has a few different default settings, including “loud,” “medium,” and “zen mode.” Turning the dial all the way down skips comments completely, while leaving the extension on full blast means the content you see on Facebook, Reddit, Twitter, or YouTube won’t be filtered at all. Putting the dial somewhere in the middle gives users more granular control over which comments, insults, profanity, etc. they want to see as the Perspective API continues to learn and improve its toxicity modeling.
The team stressed that Perspective’s ML is imperfect and experimental; hence why this is not an official Google product. Perspective still misses and miscategorizes toxic and non-toxic comments. Jigsaw has gone through years of trial-and-error in developing and testing a tool designed for the inherently subjective task of classifying the emotional impact of language.
The Perspective team hopes this crowdsourced Chrome experiment will help improve the technology, and that the open-source nature of Tune will inspire developers and show how AI can improve online conversations.
“There are other examples of Perspective’s technology being used this way,” said Adams. “There’s an extension to promote the movie Wonder about a kid getting bullied. That extension uses Perspective to let kids browse the internet and cover up abusive comments with high toxicity scores and replaces them with a motivational message you can view if you want or skip. Tune is just a more elegant, open-source version of that.”
Jigsaw also stressed that Tune isn’t meant to be a solution for direct targets of harassment, for whom blocking abusive comments might be detrimental to their safety. The more philosophical question with this kind of technology, especially when brought directly to users through a browser extension, is whether tuning out content or comments you don’t want to see improves conversations or further fragments them.
Social media allows people to surround themselves with like-minded individuals who largely confirm rather than challenge beliefs. Perspective and Tune are designed to bring people of all political and idealogical stripes back into conversations, according to Adams, by giving them a space to meaningfully disagree online.
But if you make it even easier for people to tune out the online noise they don’t like—even if a lot of it is hate speech, harassment, and trolls—these sorts of AI tools have the potential to make the internet even more of an echo chamber.
You can download the extension from the Chrome Web Store, or check out the source code and documentation on GitHub.