Tracing Semantic Shifts in Neo-Latin Texts Using Word Embeddings

Tracing Semantic Shifts in Neo-Latin Texts Using Word Embeddings Barbara McGillivray1and Valentina Lunardi2 1King’s College London, 2UCLA Digital Neo-Latin studies: ideas and perspectives University of Aarhus 24-25 September 2025 Background ●Neo-Latin is rich in terminological innovation (science, theology, law). ○Neologisms of form ○Neologisms of sense ●Classical words take on new meanings (e.g., torcular, monitor, syringa). ○Semantic neologisms played a pragmatic and strategic role, enabling early modern authors to extend the expressive range of Latin without abandoning its classical legacy ●Traditional lexica do not track these shifts quantitatively. ●Embedding-based models can offer new insights into meaning evolution. ●Very little research on embeddings models on neoLatin (Bloem et al. 2020) Semantic change research ●Numerous, varied, and often extralinguistic motivations ●Traditional methods: Tracing the meaning of each individual word within a language across time through close-reading of texts → slow and non-scalable! ●Each language has tens of thousands of words, making it difficult to find trends of change (and therefore predictability) How can embedding methods deepen our understanding of Neo-Latin’s semantics? Corpus ●LatinISE (McGillivray & Kilgarriff 2012): approx. 13 million tokens ●Semi-automatically lemmatised and PoS tagged ●Texts from IntraText digital library https://www.intratext.com/ The distributional hypothesis “You shall know a word by the company it keeps” (J. R. Firth, Selected Papers, 1957) “The degree of semantic similarity between two linguistic expressions A and B is a function of the similarity of the linguistic contexts in which A and B can appear” (Lenci 2008: 3) Word vectors Word vectors we'd eaten all these strawberries oh dear oh yeah Context window Methods: static word embeddings ●Parameters: ○Window size for co-occurrence ○Minimum frequency of a word for inclusion in training ○Epochs ○Learning rate ○ … ●In count vectors models, coordinate values = frequency counts ●In embedding models, coordinate values are not easily interpreted Periodisation Archaic and Classical (Up to 150 CE) | Post-classical (150–500 CE) | Mediaeval (500–1400 CE) | Neo Latin (After 1400 CE) Methods: contextualised word embeddings 1. Train vectors from different temporal subcorpora 2. Compare vectors of the same word 3. Look at how vectors of a target word relate to other vectors in the semantic space 4. Embeddings which are “closest” to a target embedding are known as “neighbours” Parameters ●Starting point: Latin BERT (Bamman and Burns 2020) ●Rather than training from scratch, performed domain fine-tuning (i.e., further training on a smaller corpus) ○We fine-tuned Latin BERT on LatinISE ●Parameters for fine tuning : ○Learning rate: 5e-5, 3e-5, 1e-5 –a slower learning rate yielded better results ○Epochs: 2, 3, 4 –fewer epochs were best to avoid overfitting pastor Pastoral / rural Christian / clerical Social / administrative t0 t1 t2 t3 lupus grex grex praedicator custos lupus lupus grex uenator custos custos custos grex rector rector rector canis grex doctor doctor pecus custos magister magister latro canis praesul praesul leo leo latro latro rector magister pasco pasco taurus taurus sacerdos sacerdos serpens pecus doctor presbyter bos comes magister praefectus comes pasco sacerdos administer Pastor: t0 and t1 neighbours Pastor: t2 and t3 neighbours Pastor: t0 and t1 clusters Pastor: t2 and t3 clusters Conclusions ●Semantic evolution is visible across periods: words like pastor and sacerdos shift from rural/pagan contexts to Christian/clerical contexts. ●Lexical neighbours reflect cultural change: Early periods show pagan rituals and agrarian associations, later periods show Christian liturgy and clerical hierarchy. ●Gradual and cumulative change: new neighbours appear over time, older ones fade, highlighting lexical turnover and semantic expansion. The COALA project COALA: tracing meaning over 2000 years civitas ‘(Roman) citizenship’ ‘The citizens’ ‘The city’ Large-scale annotation of word meaning in context 0 50 100 Time Citizenship Citizens City Quantitative analysis of meaning over two millennia 6. COALA