'The Big Delete:' Inside Facebook's Crackdown in Germany

Days before Germany's federal elections, Facebook took what it called an unprecedented step: the removal of a series of accounts that worked together to spread COVID-19 misinformation and encourage violent responses to COVID restrictions.

The crackdown, announced Sept. 16, was the first use of Facebook's new "coordinated social harm" policy aimed at stopping not state-sponsored disinformation campaigns but otherwise typical users who have mounted an increasingly sophisticated effort to sidestep rules on hate speech or misinformation.

In the case of the German network, the nearly 150 accounts, pages and groups were linked to the so-called Querdenken movement, a loose coalition that has protested lockdown measures in Germany and includes vaccine and mask opponents, conspiracy theorists and some far-right extremists.

Facebook touted the move as an innovative response to potentially harmful content; far-right commenters condemned it as censorship. But a review of the content that was removed — as well as the many more Querdenken posts that are still available — reveals Facebook's action to be modest at best. At worst, critics say, it could have been a ploy to counter complaints that it doesn't do enough to stop harmful content.

"This action appears rather to be motivated by Facebook's desire to demonstrate action to policymakers in the days before an election, not a comprehensive effort to serve the public," concluded researchers at Reset, a U.K.-based nonprofit that has criticized social media's role in democratic discourse.

Facebook regularly updates journalists about accounts it removes under policies banning "coordinated inauthentic behavior," a term it created in 2018 to describe groups or people who work together to mislead others. Since then, it has removed thousands of accounts, mostly what it said were bad actors attempting to interfere in elections and politics in countries around the world.

But there were constraints, since not all harmful behavior on Facebook is "inauthentic"; there are plenty of perfectly authentic groups using social media to incite violence, spread misinformation and hate. So the company was limited by its policy on what it could take down.

But even with the new rule, a problem remains with the takedowns: they don't make it clear what harmful material remains up on Facebook, making it difficult to determine just what the social network is accomplishing.

Case in point: the Querdenken network. Reset had already been monitoring the accounts removed by Facebook and issued a report that concluded only a small portion of content relating to Querdenken was taken down while many similar posts were allowed to stay up.

The dangers of COVID-19 extremism were underscored days after Facebook's announcement when a young German gas station worker was fatally shot by a man who had refused to wear a mask. The suspect followed several far-right users on Twitter and had expressed negative views about immigrants and the government.

Facebook initially declined to provide examples of the Querdenken content it removed, but ultimately released four posts to the Associated Press that weren't dissimilar to content still available on Facebook. They included a post falsely stating that vaccines create new viral variants and another that wished death on police that broke up violent protests against COVID restrictions.

Reset's analysis of comments removed by Facebook found that many were actually written by people trying to rebut Querdenken arguments, and did not include misinformation.

Facebook defended its action, saying the account removals were never meant to be a blanket ban of Querdenken, but instead a carefully measured response to users who were working together to violate its rules and spread harmful content.

Facebook plans to refine and expand its use of the new policy going forward, according to David Agranovich, Facebook's director of global threat disruption.

"This is a start," he told The AP on Monday. "This is us extending our network disruptions model to address new and emerging threats."

The approach seeks to strike a balance, Agranovich said, between permitting diverse views and preventing harmful content to spread.

The new policy could represent a significant change in the platform's ability to confront harmful speech, according to Cliff Lampe, a professor of information at the University of Michigan who studies social media.

"In the past they've tried to squash cockroaches, but there are always more," he said. "You can spend all day stomping your feet and you won't get anywhere. Going after networks is a smart try."

While the removal of the Querdenken network may have been justified, it should raise questions about Facebook's role in democratic debates, said Simon Hegelich, a political scientist at the Technical University of Munich.

Hegelich said Facebook appears to be using Germany as a "test case" for the new policy.

"Facebook is really intervening in German politics," Hegelich said. "The COVID situation is one of the biggest issues in the election. They're probably right that there's a lot of misinformation on these sites, but nevertheless it's a highly political issue, and Facebook is intervening in it."

Members of the Querdenken movement reacted angrily to Facebook's decision, but many also expressed a lack of surprise.

"The big delete continues," one supporter posted in a still-active Querdenken Facebook group, "See you on the street."