Troll Patrol: Crowdsourced Evidence of Online Abuse Against Women

Amnesty International and Element AI’s Troll Patrol set out to turn women’s long-standing accounts of Twitter abuse into evidence that could be examined at scale. The 2017 study focused on 778 women politicians and journalists in the United Kingdom and United States. More than 6,500 volunteers from 150 countries helped label more than 200,000 tweets. Using that labelled sample and further data analysis, the researchers estimated that about 1.1 million “abusive or problematic” tweets mentioned these women during the year—an average of one every 30 seconds. That figure is a model-based annual extrapolation, not a live count of a tweet arriving at a fixed interval; nor is 7.1%, another headline result, a prevalence rate for all women using Twitter.

One of the study’s most important findings is that abuse was not distributed evenly across identity. Amnesty reported that Black women were 84% more likely than white women to be mentioned in abusive or problematic tweets, while women of colour overall were 34% more likely. These figures do not mean that every attack on a woman took the same racialized form, or that political affiliation was irrelevant. They show how misogyny can intersect with racism—and how people facing the sharpest combination of harms can disappear inside an overall average when platforms do not release sufficiently disaggregated data. The women studied spanned political parties and media positions; the researchers observed abuse across the political spectrum.

The method deserves as much attention as the result. Troll Patrol did not simply ask a machine to label posts: thousands of volunteers read tweets and made contextual judgments, while data science and machine learning helped extend the estimate. That collaboration joined large-scale analysis to human interpretation and gave researchers a way to challenge platforms’ claims that the scale of the problem was unknown. But a classification is not a neutral automated fact. Amnesty distinguished “abusive” content that violated platform rules from “problematic” content that could be hostile or harmful through repetition and accumulation without necessarily crossing the same policy threshold. “Problematic” therefore does not mean every post should be removed. A policy that counts only individually removable posts may miss the cumulative harm of humiliation, threats and coordinated targeting.

The figures also have limits that matter. The study concerned public figures on Twitter in two countries in 2017; it cannot directly represent ordinary users, other countries, other platforms or today’s online environment. It depended on the data available at the time and on estimates generated from a labelled sample. This is not a current prevalence survey for all social media. Its continuing value is as a historical case in evidence-making: platforms should not be the sole institutions allowed to define, withhold or selectively disclose information about harms taking place on their services. People’s experiences can shape the research question, but categories must be explained, methods made open to scrutiny and conclusions kept within the sample’s scope.

From a feminist perspective, platform governance cannot place the burden of repair on those targeted, asking them to become silent, leave or manage the harm alone. Fairer responses require verifiable transparency reports, appeal and classification systems open to external scrutiny, recognition of repeated targeting, and meaningful participation by affected communities in rule-making. Troll Patrol’s lasting value is not to grant machines the power to delete speech, but to make platform opacity a matter of public accountability. Read it alongside FemRes entries on the Feminist Principles of the Internet and the Feminist Data Manifest-No: who gets to speak, who defines harm and who controls data are parts of the same question of power.