JHU Computational Psycholinguistics: Spring 2019

Tuesdays and Thursdays, 10:30-11:45 am

Krieger 111


Tal Linzen


Office hour: Wed 3-4 pm, Krieger 243

Teaching assistant

Suhas Arehalli


Office hour: Thu 3-4 pm, Krieger 136

Section: Fri 2-3 pm

Course description

How do we understand and produce sentences in a language we speak? How do we acquire the knowledge that underlies this ability? Computational psycholinguistics seeks to address these questions using a combination of two approaches: computational models, which aim to replicate the processes that take place in the human mind; and human experiments, which are designed to test those models. The perspective we will take in this class is that models and experimental paradigms from psycholinguistics do not only advance our understanding of the cognitive science, but can also help us advance artificial intelligence and language technologies. While research in computational psycholinguistics spans all levels of linguistic structure, from speech to discourse, the focus of this class will be at the level of the sentence (syntax and semantics).

At the end of this class, you are expected to be able to:

Prerequisites: I will assume you're familiar with probability theory (e.g., Bayes' law) and are comfortable with Python programming. Experience with neural networks would be helpful. I will also assume familiarity with basic concepts in linguistics.

Course requirements

Your responsibilities for the course are:

Homework assignments: There will be four homework assignments. These assignments will involve implementing computational models discussed in class. You have a budget of ten late days to be used at your discretion over the course of the semester, for any reason (e.g., illness); you do not need to ask for permission to use them. Use your late days wisely: once the budget has been exhausted, late assignments will receive a grade of 0.

Commentary grading: We will use the follow three-point key to grade your commentaries:

Half point adjustments will be made to account for work that does not quite fit in each of the categories, either up or down. Note that when you present an idea, we want to see at least some justification for that idea. We will not grade you on some notion of the "validity" or "reasonableness" of your justifications or proposals, but just don't say something like "they should try using 345-gram models" or "they should have used a different corpus" without stating why it's interesting or why it matters. Graduate Students will be graded out of 3 points, undergrads out of 2. That means that undergrads are not responsible for doing anything other than demonstrating that they have read and (at least vaguely) understood what's going on. You are more than welcome to get the 3rd point though - it is extra credit.

Laptop policy: Cognitive scientists have found that laptop use in the classroom can lead to lower test scores:

Raviza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2016). Logged in and zoned out: How laptop Internet use relates to classroom learning. Psychological Science, 28(2), 171–180.

We recommend that you avoid using your laptop in class, except for activities that are directly related to the class (e.g., following a Jupyter notebook).

Final project: You will be expected to write a final project in groups of two. Interdisciplinary groups made up of students whose backgrounds complement each other are particularly encouraged. The timeline for the project is:

Piazza: We will be using a Piazza site to make announcements and answer questions. Soon all enrolled students should receive an invitation to join the Piazza site. Alternatively, you can add yourself to the site.

Readings: There is no required textbook. All of the readings will be available on Piazza.

Many of the readings are from the draft third edition of Jurafsky and Martin's textbook Speech and Language Processing. Page numbers and chapters refer to the September 23, 2018 version (these chapters are also available on Piazza).

An optional resource to supplement the readings is Jacob Eisenstein's new NLP textbook (also work in progress).

Anxiety, Stress and Mental Health

If you are struggling with anxiety, stress, depression or other mental health related concerns, please consider visiting the JHU Counseling Center. If you are concerned about a friend, please encourage that person to seek out their services. The Counseling Center is located at 3003 North Charles Street in Suite S-200 and can be reached at 410-516-8278 and online.

Ethics policy

The strength of the university depends on academic and personal integrity. In this course, you must be honest and truthful. Ethical violations include cheating on exams, plagiarism, reuse of assignments, improper use of the Internet and electronic devices, unauthorized collaboration, alteration of graded assignments, forgery and falsification, lying, facilitating academic dishonesty, and unfair competition. Please report any ethics violations you witness to the instructor. You may consult the associate dean of student affairs and/or the chairman of the Ethics Board beforehand. See also the guide on ``Academic Ethics for Undergraduates'' and the Ethics Board Web site. In particular:

Do not cheat. You are encouraged to talk with other students about the content of the course, and to use written material (e.g., the slides and books, external websites, articles, newspaper/magazine stories) as sources, but your written work must be original to you, with the exception of short quotes that are clearly indicated as such (see next paragraph).

Do not plagiarize. If you quote directly from a book or other resource, please indicate this with quotes ("...") and a parenthesized citation after the quoted material; in any case, do not quote extensively from other sources. If you are simply paraphrasing a portion of a resource, leave off the quotes but keep the citation. Use a simple format for citations, for example: "human language syntax is not regular (Chomsky 1957: pages xxx-xxx)".

Disability services

Any student with a disability who may need accommodations in this class should obtain an accommodation letter from Student Disability Services, studentdisabilityservices@jhu.edu, 385 Garland, (410) 516-4720. Please bring it to our attention as early as possible so we can do the best we can to accommodate your needs.

Course outline

The topics and readings may change during the semester, depending on our rate of progress and interests.

Probabilistic prediction

Human parsing

Knowledge of grammar

Computational parsers

Probabilistic models of human parsing

Word vector representations

Syntax in neural networks

Pragmatics as inference

Information, communication and the noisy channel

Syntactic priming and adaptation

Memory and sentence processing


Extra credit: There will be no individual extra credit opportunities.

Undergraduate grade composition: Homeworks: 40%; responses: 30%; project: 20%; participation: 10%.

Graduate grade composition: Homeworks: 30%; responses: 20%; project: 40%; participation: 10%.

Letter grades: We will use the following key to assign letter grades:

Number Letter