Computational Psycholinguistics: Fall 2019

Tuesdays and Thursdays, 1:30-2:45 pm

Krieger 111

Instructor

Tal Linzen

tal.linzen@jhu.edu

Office hour: Tuesday 3-4:30 pm, Krieger 243, by appointment only (see below)

Teaching assistant

TBD

Office hour: TBD

Lab: Fri 3:45-5 pm, Krieger 111

Course description

How do we understand and produce sentences in a language we speak? How do we acquire the knowledge that underlies this ability? Computational psycholinguistics seeks to address these questions using a combination of two approaches: computational models, which aim to replicate the processes that take place in the human mind; and human experiments, which are designed to test those models. The perspective we will take in this class is that models and experimental paradigms from psycholinguistics do not only advance our understanding of the cognitive science, but can also help us advance artificial intelligence and language technologies. While research in computational psycholinguistics spans all levels of linguistic structure, from speech to discourse, the focus of this class will be at the level of the sentence (syntax and semantics).

At the end of this class, you are expected to be able to:

Prerequisites: I will assume you're familiar with probability theory (e.g., Bayes' law) and are comfortable with Python programming. I will also assume familiarity with basic concepts in linguistics. Experience with neural networks would be helpful but isn't essential.

Attendence: As this class is designed as a discussion based class, all students are expected to attend all of the meetings of the class. Please email me in advance if you need to miss a meeting for religious, health or any other valid reason. To repeat, do not come to class if you're sick, and you do not need to bring a doctor's note; but do email me in advance to let me know you'll be missing class. Repeated unexplained absence will have consequences beyond the participation grade and may result in failure in the class.

Participation: You are expected to engage in class discussion: ask questions, make comments and answer the instructors' questions. Make sure not to dominate the discussion, however: give space to all of the students to participate.

Lab: The class will be accompanied by weekly lab sessions led by the Teaching Assistant. The goals of the lab are to reinforce the linguistic, mathematical and computational concepts covered in the lecture, and to provide hands-on technical introduction to the software tools that are essential for successful completion of the homework assignments and class project. All students are expected to enroll in the lab; exceptions will be granted by the professor on a case-by-case basis (for example, to students who can demonstrate existing research experience in computational linguistics / NLP).

Office hours: If you'd like to attend my office hour, please sign up for a slot on this spreadsheet; do not show up without an appointment. To maximize access to office hours, the timing of the office hour may change from week to week (if there is sufficient demand). Please let me know if you're unable to attend my office hour due to a conflict and I'll schedule it at a different time the following week. My office hour is most appropriate for conceptual questions about course material and computational cognitive science more generally; technical issues and questions about the homework are best discussed in the lab section or the TA's office hour.

Course requirements

Your responsibilities for the course are:

Homework assignments: There will be four homework assignments. These assignments will involve implementing computational models discussed in class. You have a budget of ten late days to be used at your discretion over the course of the semester, for any reason (e.g., illness); you do not need to ask for permission to use them. Use your late days wisely: once the budget has been exhausted, any late assignments will receive at most half of the possible points.

Reactions to the readings: Each student will be expected to post a short question or comment on each of the readings before the class in which the reading is discussed. The reactions are expected to demonstrate that you have read and thought about the article. You can skip up to five reactions without penalty; after that, every missed reaction will be penalized with a single point. Particularly thoughtful reactions will be rewarded with an extra credit point, up to a maximum of 5 points.

Laptop policy: Cognitive scientists have found that laptop use in the classroom can lead to lower test scores:

Raviza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2016). Logged in and zoned out: How laptop Internet use relates to classroom learning. Psychological Scjience, 28(2), 171–180.

See also the New York Times opinion piece, Laptops Are Great. But Not During a Lecture or a Meeting.

We recommend that you avoid using your laptop in class, except for activities that are directly related to the class (e.g., following a Jupyter notebook in lab sessions).

Piazza: We will be using a Piazza site to make announcements and answer questions. Soon all enrolled students should receive an invitation to join the Piazza site. Alternatively, you can add yourself to the site.

Readings: There is no required textbook. All of the readings will be available on Piazza.

Many of the readings are from the draft third edition of Jurafsky and Martin's textbook Speech and Language Processing. Page numbers and chapters refer to the September 23, 2018 version (these chapters are also available on Piazza).

An optional resource to supplement the readings is Jacob Eisenstein's new NLP textbook (also work in progress).

Final project

Graduate students will be expected to write a final paper reporting on an original research project in the area of computational psycholinguistics. The project is expected be at a level that can lead to a conference submission.

This project will make up 40% of the grade. The timeline for the project is as follows (all deadlines are by 6 pm Eastern time):

Proposal: The proposal should be up to two pages including references, and include the following parts:

Final report: The final report should be up to six pages including references. The report is expected to include the following content (not necessarily as distinct parts):

Format: Please use the Cognitive Science Society (CogSci) LaTeX template for both the proposal and the final report. Overleaf's Learn LaTeX in 30 minutes tutorial may be helpful to students who haven't used LaTeX before.

Anxiety, Stress and Mental Health

If you are struggling with anxiety, stress, depression or other mental health related concerns, please consider visiting the JHU Counseling Center. If you are concerned about a friend, please encourage that person to seek out their services. The Counseling Center is located at 3003 North Charles Street in Suite S-200 and can be reached at 410-516-8278 and online.

Ethics policy

The strength of the university depends on academic and personal integrity. In this course, you must be honest and truthful. Ethical violations include cheating on exams, plagiarism, reuse of assignments, improper use of the Internet and electronic devices, unauthorized collaboration, alteration of graded assignments, forgery and falsification, lying, facilitating academic dishonesty, and unfair competition. Please report any ethics violations you witness to the instructor. You may consult the associate dean of student affairs and/or the chairman of the Ethics Board beforehand. See also the guide on ``Academic Ethics for Undergraduates'' and the Ethics Board Web site. In particular:

Do not cheat. You are encouraged to talk with other students about the content of the course, and to use written material (e.g., the slides and books, external websites, articles, newspaper/magazine stories) as sources, but your written work must be original to you, with the exception of short quotes that are clearly indicated as such (see next paragraph).

Do not plagiarize. If you quote directly from a book or other resource, please indicate this with quotes ("...") and a parenthesized citation after the quoted material; in any case, do not quote extensively from other sources. If you are simply paraphrasing a portion of a resource, leave off the quotes but keep the citation. Use a simple format for citations, for example: "human language syntax is not regular (Chomsky 1957: pages xxx-xxx)".

Disability services

Any student with a disability who may need accommodations in this class should obtain an accommodation letter from Student Disability Services, studentdisabilityservices@jhu.edu, 385 Garland, (410) 516-4720. Please bring it to our attention as early as possible so we can do the best we can to accommodate your needs.

Course outline

The topics and readings may change during the semester, depending on our rate of progress and interests.

Probabilistic prediction

Human parsing

Knowledge of grammar

Computational parsers

Probabilistic models of human parsing

Word vector representations

Syntax in neural networks

Pragmatics as inference

Information, communication and the noisy channel

Syntactic priming and adaptation

Memory and sentence processing

Grading

Extra credit: There will be no individual extra credit opportunities.

Undergraduate grade composition:

Graduate grade composition:

Letter grades: We will use the following key to assign letter grades:

Number Letter
97–100A+
93–96A
90–92A-
87–89B+
83–86B
80–82B-
77–79C+
73–76C
70–72C-
60–69D
0–59F