Instructor: Prof. Kyle Mahowald (he/him)
Meeting times: Tues. 2pm-5pm
Location: RLP computational linguistics lab
Office hours: Friday 10am-11:30am (on Zoom)
Large Language Models (LLMs) like ChatGPT have made remarkable strides in modeling language, particularly in producing fluid and grammatical text. It has been claimed that their apparent success on formal linguistic tasks (e.g., syntax) is illusory and/or unlike the way that humans solve such tasks. It has also been claimed that such models are still deficient on functional tasks that involve logical reasoning, commonsense reasoning, and world knowledge. We consider both of these claims, with a focus on the recent literature on LLMs in NLP, linguistics, cognitive science, and philosophy. Students will complete a final project that explores the linguistic and cognitive capacities of LLMS or investigates how they achieve such capacities.
1) You will learn to do analyses studying the behavior of language models. LLMs are used in a wide range of applications, and there is a growing field seeking to understand their behavior. You will get exposure to that field and, more importantly, get practical experience doing your own projects exploring the space. There will be a focus on not just reading papers, but on thinking about the methods used, their advantages and disadvantages, and what’s involved in doing your own work in this space.
2) You will learn about linguistic structure. We will survey some ideas about the kinds of structures (hierachical grammars, abstraction, etc.) needed to learn language and discuss whether we should expect computational models to learn the same kinds of structure. Those with less experience in linguistics will learn some of the key concepts, and those with more experience in linguistics will develop these further.
3) You will gain general experience designing and completing a research project in the neural network analysis space. When we are reading papers, I want you to be thinking about how the papers relate to your own interests and, together, we will find an interesting research project for you to do.
4) You will get practice presenting your research, via the various project presentations.
Twice during the term, you will take the lead presenting a paper to the class. This can be relatively informal, but you are encouraged to have some slides or a handout prepared.
When you are not presenting, I will ask you to come to class with at least one question about the paper.
At least once during the term, you will come to class with a “fun little project”. This can be some kind of data exploration or idea that takes you maybe a couple hours to put together and that relates to the themes of the course.
The final project is open ended, and I want it to be something that you are interested in and that helps you further your research goals. We can work together to find something. There will be an intermediate step, where we do project proposals and can brainstorm as a group.
We will use Slack for course discussions. You will post a question/comment/idea each week, in response to the paper(s) we read.
| Activity | Weight |
|---|---|
| Fun Little Project | 10% |
| 2 paper leads | 10% |
| Class participation (including class attendance and weekly Slack posts) | 20% |
| Final project (40% written, 10% proposal, 10% presentation) | 60% |
Week 8 10/10: LLMs and Grounding; Ray Mooney visits
Week 12 11/7: Nathan Schneider visits, Constructions and LLMs Redux
Week 13 11/14: Final project presentations
A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.
Class recordings are reserved only for students in this class for educational purposes and are protected under FERPA. The recordings should not be shared outside the class in any form. Violation of this restriction by a student could lead to Student Misconduct proceedings.
You are encouraged to discuss assignments with classmates. But all written work must be your own. Students caught cheating will automatically fail the course. If in doubt, ask the instructor.
No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class unless you have my explicit, written permission. Unauthorized sharing of materials promotes cheating. It is a violation of the University’s Student Honor Code and an act of academic dishonesty. I am well aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Student Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in sanctions, including failure in the course.
The university is committed to creating an accessible and inclusive learning environment consistent with university policy and federal and state law. Please let me know if you experience any barriers to learning so I can work with you to ensure you have equal opportunity to participate fully in this course. If you are a student with a disability, or think you may have a disability, and need accommodations please contact Services for Students with Disabilities (SSD). Please refer to SSD’s website for contact and more information: http://diversity.utexas.edu/disability/. If you are already registered with SSD, please deliver your Accommodation Letter to me as early as possible in the semester so we can discuss your approved accommodations and needs in this course.
The Counseling and Mental Health Center serves UT’s diverse campus community by providing high quality, innovative and culturally informed mental health programs and services that enhance and support students’ well- being, academic and life goals. To learn more about your counseling and mental health options, call CMHC at (512) 471-3515.
If you are experiencing a mental health crisis, call the CMHC Crisis Line 24/7 at (512) 471-2255.
Did you know that more than one-third of UT undergraduate students use the Sanger Learning Centereach year to improve their academic performance? All students are welcome totake advantage of Sanger Center’s classes and workshops, private learning specialist appointments, peer academic coaching,and tutoring for more than 70 courses in 15 different subject areas. For more information, please visit http://www.utexas.edu/ugs/slcor call 512-471-3614 (JES A332).
Each student in the course is expected to abide by the University of Texas Honor Code: “As a student of The University of Texas at Austin, I shall abide by the core values of the University and uphold academic integrity.”Plagiarism is taken very seriouslyat UT.Therefore, if you use words or ideas that are not your own (or that you have used in previous class), you must cite your sources. Otherwise you will be guilty of plagiarism and subject to academic disciplinary action, including failure of the course. You are responsible for understanding UT’s Academic Honesty and the University Honor Code which can be found at the following web address: https://deanofstudents.utexas.edu/conduct/standardsofconduct.php