Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification

Joy Buolamwini and Timnit Gebru’s 2018 paper asks whether an impressive overall accuracy can conceal errors concentrated on particular groups. Its task is commercial gender classification from facial images, rather than determining someone’s identity or measuring every use of facial recognition. That distinction matters: a benchmark result can expose an unequal distribution of errors without establishing how every surveillance system performs.

Build a benchmark that can reveal differences

The researchers examined the composition of IJB-A and Adience and constructed the Pilot Parliaments Benchmark, or PPB, containing 1,270 parliamentarians from six African and European countries. Parliamentary portraits offered identifiable public figures and comparatively controlled images. The selection improved representation for the evaluation, but does not make the sample representative of every population, camera condition or social setting.

The original paper, sections 3.4–3.5 and table 3, groups Fitzpatrick skin types I–III as lighter and IV–VI as darker. It crosses those groups with perceived female and male labels. These labels follow the binary outputs of the commercial systems; the authors explicitly discuss their inability to capture the complexity of gender and transgender identities. A dermatologist’s contribution to skin-type labeling does not turn gender into a biological fact readable from a face.

Averages distribute reassurance unevenly

The audit evaluated Microsoft, IBM and Face++ classifiers. Darker female subjects had the highest error rates across the tested systems, ranging from 20.8% to 34.7%; the maximum error for lighter male subjects was 0.8%. These are results for particular systems, versions, images and subgroup definitions in this study, not current error rates for all AI. A comparison that only reports performance for women or for darker faces separately can still conceal the intersectional pattern.

The feminist contribution is therefore methodological as well as numerical. The question is who disappears inside the average, whose face defines ordinary performance, and which failures are tolerated as marginal exceptions. Stratified evaluation makes claims of universal reliability contestable. It also shows why adding more people to a data set is not enough unless one examines how they are distributed, labeled and tested.

Better classification is not the same as justified classification

The benchmark also has limits. Official portraits are more controlled than many images captured in streets or other environments; skin-type estimation from photographs introduces uncertainty. Grouping six skin types into two categories makes an evaluation possible while reducing differences within each group. None of these limitations cancels the disparities, but they prevent an experimental result from becoming a universal account of bodies or social identities.

There is a further political question: even a more accurate classifier may be inappropriate when people cannot refuse the classification or challenge its use. The paper’s evidence does not by itself decide whether a particular institution should deploy such a tool. Nor does a 2018 publication prove the timing or causes of later company restrictions on police sales. Technical findings, subsequent advocacy and corporate decisions need their own evidence.

Read the study alongside Buolamwini’s public talk, which offers an accessible account of encountering the problem, and Data Feminism, which widens the questions of power, context and labor. Design Justice then asks who can change the system itself. These connections keep two questions open together: who bears the errors, and who has authority to classify anyone in the first place?