How Urdu news writes about women and men, measured four ways: grammar, adjectives, word embeddings and topics.
14,289 articles
15,530 articles
At a glance
Larger bodies of work can carry linguistic bias, and newspapers are an excellent example. Are women written about as people who "do" things, or as people things happen "to"? Are certain words used typically around women that aren't used for men? This phenomenon has been studied previously in English, and emerging studies have done so for Arabic, Turkish and Hindi. Even though Urdu is spoken by over 250 million people worldwide, it has received much less attention.
In this analysis, we ask a simple question: how do the words used around men and women in Urdu news differ? We answered this question by measuring four things.
Our articles come from a public dataset of Urdu news, containing 111,861 articles across four categories: Business & Economics, Science & Technology, Entertainment, and Sports. Each article was flagged using a list of female and male terms. 14,289 articles contained a female term and 15,530 a male term; 7,060 contained both, leaving 7,229 female-only and 8,470 male-only articles. So only about 20% of the original corpus was used for this analysis.
Articles flagged against the list of female and male terms.
Female terms appear in 14,289 articles in total and male terms in 15,530; the 7,060 articles that mention both are counted in each.
200 articles from the female-term set and 200 from the male-term set were sampled and parsed with Stanza's Urdu pipeline (no parsing failures). For each gendered term, we recorded whether it was used as a grammatical subject or object. Only documents with one countable role were assigned an "agency rate" — subjects ÷ (subjects + objects/passives) — which left about 62 female and 49 male articles. The group difference in mean document-level rate was tested with a 10,000-permutation test.
Adjectives attached to female and male terms were counted (102 female and 53 male modifier tokens; 64 vs. 33 distinct words) and compared with a log-odds score. A positive score means a word is more distinctive of female terms; a negative score, of male terms. At this sample size, these statistics are best read descriptively.
A skip-gram Word2Vec model was trained on the full corpus with a min_count of 20. Female and male attribute sets (47 and 45 terms) were reduced by the frequency cutoff to 37 and 34 in-vocabulary terms. We defined five categories to test — Leadership/Power, Domestic/Family, Crime/Legal, Victimhood/Vulnerability, and Professional/Career — for 94 concept tests in total. For each test we calculated the mean cosine similarity of the concept to each attribute set, then the difference Δ = female − male. Significance came from a randomization test that reshuffles which terms carry a male or female label, followed by a Benjamini–Hochberg FDR correction across all 94 tests at α = 0.05. Randomization p-values shift by a few thousandths between runs.
BERTopic, with UMAP dimensionality reduction, was run separately on 500 female-term and 500 male-term articles. Outlier documents were reassigned so every article belonged to a topic. Each topic is a cluster of frequently co-occurring keywords, which we labelled in plain English and then grouped into six broader themes. Since labelling is up to interpretation, the individual topics are listed in the appendix below.
Across all parsed mentions, female terms were grammatical subjects 48 times and objects or passives 55 times (46.6% agentive). For male terms, this was 39 and 30 (56.5%).
Share of gendered mentions that were grammatical subjects, all parsed mentions pooled (103 female, 69 male).
Solid: grammatical subjectPale: object or passive
Treating each mention as independent, this gap is not significant (Fisher's exact p ≈ 0.22). Mentions inside one article aren't independent, so the document-level test below is the better one.
At the document level, the mean agency rate was 0.44 for female articles and 0.61 for male articles, a difference of −0.173 (permutation p = 0.050). Nearly half of the female documents — 30 of 62 — contained only object or passive mentions, while over half of the male documents — 26 of 49 — contained only subject mentions.
A bootstrap on the same 111 documents gives a 95% interval for the difference of roughly −0.34 to −0.005, which only just excludes zero. If each mention is treated as independent, the comparison is not significant (Fisher's exact p ≈ 0.22). Mentions inside an article aren't independent, so the document-level test is the better one, but this remains a suggestive result given the small sample.
Mean document-level agency rate: subjects ÷ (subjects + objects or passives).
0.44
Female-term articles (62 usable)
0.61
Male-term articles (49 usable)
−0.173
Difference, permutation p = 0.050
Dot: observed difference (female minus male). Bar: 95% bootstrap interval, −0.34 to −0.005, over the same 111 documents.
The biggest problem we ran into here was that gender is built into Urdu grammar. Adjectives that scored as strongly feminine or masculine, such as بڑی and بڑا ("big," feminine and masculine forms), often just reflected grammatical agreement with whoever was being described, not any real bias. Since there's no list of gendered Urdu terms we could use to exclude, we were forced to rely on manually created stopword lists, which were not perfect — so the hatched bars below are exactly this grammatical-agreement artifact, not a distinctive-usage finding.
Number of times each adjective was attached to a female or a male term.
Hatched: gender-agreement form, likely an artifact
Female terms102 modifier tokens, 64 distinct words
Male terms53 modifier tokens, 33 distinct words
Both sides use the same scale. English glosses are approximate.
By log-odds score, female terms leaned towards words such as سوتیلی ("step-"), نوجوان ("young"), and خوبصورت ("beautiful"), as opposed to the more ordinary male terms, which included مبتلا ("afflicted"), شفیق ("kind"), and انجہانی ("the late," i.e. deceased). However, these correlations did not reach FDR significance, so this should be read as descriptive insight rather than confirmed.
Most distinctive of female terms
Most distinctive of male terms
Across all 94 tests, we found that our average difference was −0.011, and 63 of 94 concepts sit closer to the male attribute set. Leadership/Power (17 of 20) and Crime/Legal (18 of 19) are almost exclusively male, but individual effect sizes are small, so most words do not survive correction. Domestic/Family is the only category where more concepts lean female than male (12 of 19).
Number of concepts in each category that sit closer to the female or to the male attribute set.
Dark line marks an even split. Categories are ordered from most female-leaning to most male-leaning.
Primarily, six associations survive correction (q < 0.05). Terms such as چیئرمین ("chairman"), کاروبار ("business"), دفتر ("office") and روزگار ("employment") — all professional, career-oriented terms — were male, with 0.095–0.221 similarity. Conversely, the two female terms were گھریلو ("domestic") and مظلوم ("oppressed/wronged"), which relate to family and victimhood themes respectively, with 0.272–0.382 similarity.
Cosine similarity of each word to the female and male attribute sets. Δ = female − male; all six survive FDR correction (q < 0.05).
Similarity to female termsSimilarity to male terms
Closer to male terms
Closer to female terms
Similarities ranged 0.095–0.221 on the male-leaning side and 0.272–0.382 on the female-leaning side. The tests use single-run embeddings; see limitations.
Our topic modelling results found that film and celebrity coverage is the largest theme in both samples (34% female, 32% male), followed by sport (21% vs. 23%). The differences are present in the smaller themes. Articles mentioning women include more crime stories (11% vs. 6%), driven by harassment and assault cases, as well as fashion and culture. Articles mentioning men spread into a broader bucket of general Pakistan news, relating to technology, science, economics and COVID-19.
Share of each sample of 500 articles, by theme.
Female-term articlesMale-term articles
BERTopic run separately on 500 female-term and 500 male-term articles; every article, including outliers, was assigned to a topic. Topics were labelled by hand and grouped into themes.
Female terms are more often the object or passive of a sentence.
Female terms sit nearer household and victimhood vocabulary.
Women appear more often in stories about harassment and courts.
Although each of our results is modest alone, they broadly point to a trend of women being under-represented in public-sphere roles and often being framed as marginalized or as victims. Women are often mentioned more in object or passive roles (agency test), sit nearer household and victimhood vocabulary (Word2Vec), and appear more often in stories about harassment and courts (topic modelling).
However, the male attribute set may simply be the baseline here. The two attribute sets differ in size, term mix and frequency, and embedding similarity tends to track frequency. The average Δ of −0.011 could be a baseline shift rather than "everything is more male." A useful next step is to centre Δ on the mean and re-test.