Category Archives: Computational linguistics

Frank colloquium Friday Oct 11 at 3:30

Bob Frank, Yale University, will present “Inductive Bias in Language Acquisition: UG vs. Deep Learning” in the Linguistics colloquium series at 3:30 Fri. Oct 11. An abstract follows. All are welcome!

Abstract: Generative approaches to language acquisition emphasize the need for language-specific inductive bias, Universal Grammar (UG), to guide learners in the face of limited data. In contrast, computational models of language learning, particularly those rooted in contemporary neural network models, have achieved high levels of performance on practical NLP tasks, largely without the imposition of any such bias. While UG-based approaches have led to important insights into the stages and processes underlying language acquisition, they have not yielded a concrete, mechanistic model of the process by which language is learned. At the same time, practical computational models have not been widely tested with respect to their ability to extract linguistically significant generalizations from training data. As a result the ability of such systems to face the challenges identified in the generative tradition remains unproven. In this talk, I will review several experiments that explore the ability of network models to take on such challenges. Looking at question formation and subject-verb agreement, we find that there is considerable variety in the degree to which network architectures are capable of learning significant grammatical generalizations through gradient descent learning, suggesting that the architectures themselves may be able to impose some of the necessary bias that is often assumed to motivate the need for UG. Inadequacies remain in the generalizations acquired, however, which points to the need for hybrid models that integrate language specific information into network models.

Language and Music Workshop this Sunday May 12th

The UMass Amherst Department of Linguistics and the Department of Music and Dance, with additional support from the Interdisciplinary Studies Institute, will host a Language and Music Workshop on the afternoon of Sunday May 12th. The event will take place from noon until 5:45 in N400 in the Integrative Learning Center. Parking is free in permit lots on Sunday; the ILC is at the top corner of the pond on this map.

There are five invited speakers, and five poster presentations listed below. Please join us for lunch beforehand!

Questions? Please e-mail Joe Pater at pater@umass.edu.

Schedule

Noon – Catered lunch

1:00 Bob Ladd – University of Edinburgh

Two problems in theories of tone-melody matching (Abstract)

1:45 François Dell – Centre de Recherches Linguistiques sur l’Asie Orientale (CRLAO) CNRS / EHESS, Paris

Delivery design: towards a typology (Abstract)

2:30 Laura McPherson – Dartmouth College

Tonal adaptation across musical modality: A comparison of Sambla vocal music and speech surrogates (Abstract)

3:15 Poster session (see below for a list of posters)

4:15 Christopher White – University of Massachusetts Amherst

Analogies with Language in Machine-learned Musical Grammars

5:00 Mara Breen – Mount Holyoke College

The Cat in the Hat: Musical and linguistic metric structure realization in child-directed poetry (Abstract)

5:45 Goodbye.

Posters

Ellie Abrams, Laura Gwilliams, Alec Marantz (NYU, NYU Abu Dhabi)

Tracking the building blocks of pitch perception in auditory cortex (Abstract)

Kyle Marcos Allphin, Smith College ’19

Perception of Emotional Characteristics in Diatonic Chords (Abstract)

Ahren B. Fitzroy (Mount Holyoke College, University of Massachusetts, Amherst) and Mara Breen (Mount Holyoke College)

Implicit metric structure in aprosodic productions of The Cat in the Hat modulates auditory processing (Abstract)

Bronwen Garand-Sheridan, Yale University

Sound-symbolic semantics of pitch space (Abstract)

Emily Schwitzgebel, UMass Amherst and Will Evans, UMass Amherst

Subtle Violations in Harmonic Expectancy (Abstract)

Magnuson CogSci talk at noon Wednesday in ILC N400

James Magnuson (https://magnuson.psy.uconn.edu/) will present a talk sponsored by the Five College Cognitive Science Speaker Series in ILC N400 from at noon Wednesday 27th. Pizza will be served. The title and abstract are below. All are welcome!

EARSHOT: A minimal neural network model of human speech recognition that learns to map real speech to semantic patterns

James S. Magnuson, Heejo You, Hosung Nam, Paul Allopenna, Kevin Brown, Monty Escabi, Rachel Theodore, Sahil Luthra, Monica Li, & Jay Rueckl

One of the great unsolved challenges in the cognitive and neural sciences is understanding how human listeners achieve phonetic constancy (seemingly effortless perception of a speaker’s intended consonants and vowels under typical conditions) despite a lack of invariant cues to speech sounds. Models (mathematical, neural network, or Bayesian) of human speech recognition have been essential tools in the development of theories over the last forty years. However, they have been little help in understanding phonetic constancy because most do not operate on real speech (they instead focus on mapping from a sequence of consonants and vowels to words in memory), and most do not learn. The few models that work on real speech borrow elements from automatic speech recognition (ASR), but do not achieve high accuracy and are arguably too complex to provide much theoretical insight. Over the last two decades, however, advances in deep learning have revolutionized ASR, with neural network approaches that emerged from the same framework as those used in cognitive models. These models do not offer much guidance for human speech recognition because of their complexity. Our team asked whether we could borrow minimal elements from ASR to construct a simple cognitive model that would work on real speech. The result is EARSHOT (Emulation of Auditory Recognition of Speech by Humans Over Time), a neural network trained on 1000 words produced by 10 talkers. It learns to map spectral slice inputs to sparse “pseudo-semantic” vectors via recurrent hidden units. The element we have borrowed from ASR is to use “long short-term memory” (LSTM) nodes. LSTM nodes have a memory cell and internal “gates” that allow nodes to become differentially sensitive to variable time scales. EARSHOT achieves high accuracy and moderate generalization, and exhibits human-like over-time phonological competition. Analyses of hidden units – based on approaches used in human electrocorticography – reveal that the model learns a distributed phonological code to map speech to semantics that resembles responses to speech observed in human superior temporal gyrus. I will discuss the implications for cognitive and neural theories of human speech learning and processing.

CLC Talk on Unsupervised Learning of Phrase Structure – November 15 @ 4pm

The first CLC (Computational Linguistics Community) event of the semester will be a talk on unsupervised learning of phrase structure. The talk will be at 4pm on November 15th and will take place as part of the new Neurolinguistics Reading group. All are welcome! Please see below for more details.

TITLE:
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders

AUTHORS:
Andrew Drozdov*, Pat Verga*, Mohit Yadav*, Mohit Iyyer, Andrew McCallum

ABSTRACT:
Syntax is a powerful abstraction for language understanding. Many downstream tasks require segmenting input text into meaningful constituent chunks (e.g., noun phrases or entities); more generally, models for learning semantic representations of text benefit from integrating syntax in the form of parse trees (e.g., tree-LSTMs). Supervised parsers have traditionally been used to obtain these trees, but lately interest has increased in unsupervised methods that induce syntactic representations directly from unlabeled text. To this end, we propose the deep inside-outside recursive auto-encoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Unlike many prior approaches, DIORA does not rely on supervision from auxiliary downstream tasks and is thus not constrained to particular domains. Furthermore, competing approaches do not learn explicit phrase representations along with tree structures, which limits their applicability to phrase-based tasks. Extensive experiments on unsupervised parsing, segmentation, and phrase clustering demonstrate the efficacy of our method. DIORA achieves the state of the art in unsupervised parsing (48.7 F1) on the benchmark WSJ dataset.

LOCATION:
ILC N400