Pigeon Effect
All Projects

UNGD Linguistic Evolution

Thirteen simple text measures, tracked across seventy-five years of UN General Debate speeches.

2024-08 · 1946 – 2022 · 164 views ·
NLP NLTK TextBlob Time series

Each September, heads of state and government address the UN General Assembly in its General Debate. The occasion repeats every year in the same format, which makes the speeches a simple corpus for asking whether the language of diplomacy has changed. This project is a basic first pass: thirteen easy-to-compute text measures, averaged over all speeches of each year and charted from 1946 to 2022.

Research Questions

  • Have General Debate speeches become wordier, harder to read, or more varied in vocabulary?
  • Do speakers refer to themselves, address the audience, or negate more often than they used to?
  • Does the tone, as an off-the-shelf sentiment lexicon scores it, drift over the decades?

The corpus

The speeches come from the UN General Debate Corpus, which holds every country statement since 1970; the earlier sessions were added from scanned UN records through OCR and matched by hand. Speeches are grouped by session year. Most measures have no values for 1948 and 1949.

Method

Deliberately simple. Each speech is cleaned of paragraph numbers and tokenized with NLTK. The rates are counts from short word lists (pronouns, modal and degree adverbs, negations, crisis terms, swear words, numbers) divided by the number of tokens. Lexical diversity is the moving type-token ratio over 500-token windows; readability is the Flesch Reading Ease score, with syllables from the CMU Pronouncing Dictionary; sentiment polarity and subjectivity are TextBlob's lexicon scores. Each measure is then averaged over all speeches of a year.

What these numbers can and cannot say

These are surface measures. A word list counts 'crisis' the same whether a speaker warns of one or denies it, and a general-purpose sentiment lexicon was not built for diplomatic prose, so the lines describe style, not intent. The yearly averages also mix a changing membership: the Assembly grew from 51 states to 193, so part of any trend is simply who is at the podium.

13 measures, 1946 – 2022

Rhetorical markers, readability and sentiment, and word-list rates, each charted as a yearly average with a degree-5 polynomial trend. The values are the yearly averages published with the original analysis, redrawn here. Hover any line for the exact year and value.