Is automated fact-checking the “holy grail” for this corner of journalism?
By
Instead of watching the upcoming State of the Union address with snide fact-checks from users on Twitter or other social media in the background, viewers will be able to see instantaneous fact-checking appear on their device screen as soon as President Trump utters a claim — or at least that’s the dream for Bill Adair’s team at Duke University’s Reporters’ Lab.
Adair, the fact-checking maven who founded PolitiFact and is now director of the Reporters’ Lab, is spearheading the nascent Tech & Check Cooperative to bundle automation with a number of initiatives already launched in the fact-checking/computer science sphere. The two-year project got underway in the fall and is funded with a total of $1.2 million from the Knight Foundation, the Facebook Journalism Project, and the Craig Newmark Foundation. It draws on automation and fact-checking work from researchers from the University of Texas at Arlington, the Internet Archive, MIT Media Lab, and Cal Poly-San Luis Obispo. Its live fact-check of the State of the Union Jan. 30 will be the team’s first beta test.
“If you had asked me three or four years ago how soon will you be able to do live pop-up fact checking I would have said maybe a decade…but artificial intelligence has come a long way in a short period of time,” Adair said. “We’ve created the secret sauce with our database…We see this very much as a way to bring together computer scientists and fact checkers to do this work.” The database, in case you couldn’t tell, is currently only available for Tech & Check partners.
From the announcement of the cooperative in September:
Partners in the Tech & Check Cooperative include:
- The University of Texas at Arlington, which has developed ClaimBuster, a tool that can mine lengthy transcripts for claims that fact-checkers might want to examine.
- The Internet Archive, which will help develop a “Talking Point Tracker” that will identify factual claims that are used repeatedly by politicians and pundits.
- Truth Goggles, a project created by developer Dan Schultz and the Bad Idea Factory to provide pop-up fact-checking for articles on the web.
- Digital Democracy, an initiative of the Institute for Advanced Technology and Public Policy at Cal Poly-San Luis Obispo, which will develop ways to identify factual claims from video of legislative proceedings in California.
The cooperative stems from the R&D-focused Reporters’ Lab’s Share the Facts creation in 2016. Share the Facts was built originally as an embeddable widget for news articles but has evolved to an add-on for both Google Home and Amazon’s Alexa for users to ask the devices a question about a claim made by a celebrity, politician, or other newsmaker. The audio device can reply to the listener — if the statement aligned with the fact-check has already been processed by the fact-checking partners (including PolitiFact, The Washington Post, and Gossip Cop) and is in the database. But the user has to prompt the check: “One of the weaknesses of fact checking is that you basically have to do homework — you have to look it up,” Adair noted. There are more than 12,000 fact-checks in the Share the Facts database, but automation could be a clear opportunity to incessantly add more.
For one example, “I think of ClaimBuster as a college intern who works 24/7, who can sit in the gallery of the House or the Senate, who can watch 24 hours of cable news and be a good tool at finding claims that fact checkers might want to check,” Adair said. “We’re using automation to do some tedious work for fact-checkers that humans would otherwise have to do.”
Adair largely deals with the structure of producing fact-checks, but the question of fact-checking’s impact still stands. You can create a tool to plop fact-checks on a screen like a pop-up ad, sure, but frequency or placement doesn’t necessarily equate with increasing trust or effectiveness. We know that fact checks work better if they come from acquaintances rather than strangers; what about when they come from automated prompts?
That’s part of what Truth Goggles, among many others in the fact-checking world, has been inquiring. We wrote about Dan Schultz’s project five years ago, as it focused on flagging potentially false claims at the moment of consumption. “Even if you are a nonpartisan fact-checker, you’re going to anger one or both sides, and that’s the nature of this disruptive form of journalism. And at a time when people are going into echo chambers for their information, it can be a challenge,” Schultz told us in 2012. “The one thing I would say to that is I don’t think what we’re doing is telling people what to think. We’re just trying to tell them information to consider.” (Schultz now works at the Internet Archive.)The joint attack on misinformation via computerized fact-checking is part of Tech & Check’s cooperative nature. (“This is largely a technology and journalism project. It’s not a social psychology project,” Adair said.) He sees automated fact-checking as the “holy grail” for this corner of journalism: making the interaction so instant and instinctual from both perspectives, the device providing the check and the user seeking it.
But there’s also been progress made with automating fact-checks in other spheres. We covered the Greece-based, English-language FightHoax algorithm earlier this year:
FightHoax attempts to perform text analyses of news articles. What it’s trying to offer is detection, not actual fact-checking of claims, as promotional materials claim. It isn’t able to refute or contextualize specific statements by drawing from a comprehensive database of existing fact-checks, which new automated programs like Full Fact or Factmata are working towards. The Argentinian fact-checking outlet Chequeado is also working on a beta version of an automated fact-checking software that will evaluate Spanish-language claims by comparing it to similar previously made ones already available in its large database of claims.
In an industry fraught with fact-less-ness and seeking collaboration in all corners, this teamwork might make the dream work.
“Too often when you use our [Share the Facts] apps, Alexa will say, ‘I’m sorry, I don’t have any information on that,’” Adair said. “We are working with others around the world, doing this work to make sure we all learn from each other, all have common standards, swap ideas, and overcome problems together.”
By
