From the UMassNLP mailing list – all are welcome to attend! Zoom link left out for security – please e-mail Joe Pater if you need it!

Speaker: Brian Dillon

Title: Surprisal does not explain syntactic disambiguation difficultyTime: Tuesday 10/3 (11:30-12:30)

Abstract: Prediction has been proposed as an overarching principle that explains human information processing in language and beyond.  To what degree can processing difficulty in syntactically complex sentences—one of the major concerns of psycholinguistics—be explained by predictability, as estimated using computational language models?  A precise, quantitative test of this question requires a much larger scale data collection effort than has been done in the past.  We present the Syntactic Ambiguity Processing Benchmark, a dataset of self-paced reading times from 2000 participants, who read a diverse set of complex English sentences.  This dataset makes it possible to measure processing difficulty associated with individual syntactic constructions, and even individual sentences, precisely enough to rigorously test the predictions of computational models of language comprehension.  We find that the predictions of language models with two different architectures sharply diverge from the reading time data, dramatically underpredicting processing difficulty, failing to predict relative difficulty among different syntactic ambiguous constructions, and only partially explaining item-wise variability.  These findings suggest that prediction is most likely insufficient on its own to explain human syntactic processing.

Bio: Brian Dillon is an Associate Professor in the Linguistics Department. He is a psycholinguist who specializes in adult language comprehension. He is specifically interested in the role of syntax in incremental sentence processing.

Paper: https://psyarxiv.com/z38u6/