Advanced Natural Language Processing

UCLA CS 263, Winter 2026

Lecture: M/W 4-5:50pm, Perloff Hall 1102

Instructor: Saadia Gabriel
Email: skgabrie@cs.ucla.edu
Office: Eng VI 295A
Office Hours: Mondays 1:45-2:45pm

TAs:
Genglin Liu
Email: genglinliu@g.ucla.edu
Office Hours: TBD

Yunqi Hong
Email: yunqihong@g.ucla.edu
Office Hours: TBD

Auguste Hirth
Email: ahirth@g.ucla.edu
Office Hours: TBD

Course Description: Natural language processing (NLP) enables computers to understand, process and mimic human languages. NLP techniques have been widely used in many applications, including machine translation, question answering, machine summarization, and information extraction. We will study fundamental elements and recent trends in modern NLP. Students will gain ability to apply NLP techniques in text-based applications, understand core algorithms used in NLP, critically read NLP research papers and propose new approaches to solve NLP problems.

Schedule:

Date Topic Description Assignment(s)
1/5 Week 1: Intro   We will go over the syllabus, schedule, and course expectations. We will recap fundamental pre-transformer concepts covered by introductory natural language processing courses, including distributional semantics and word embeddings, pragmatics and basics of neural networks.
1/7 Week 1: Neural Networks We will recap basics of sequence-to-sequence modeling and discuss specific neural network architectures like recurrent neural networks and transformers. We will discuss differences between types of transformer models (e.g. decoder-only vs. encoder-decoder).
1/12 Week 2: Large-scale Language Modeling We'll discuss the pre-train/post-train paradigm of large-scale language modeling. We'll walk through the evolution of pretrained transformer models, common pre-training objectives and responsible data practices.
1/14 Week 2: Guest Lecture
  • Final project group assignments out
1/21 Week 3: LLM Post-training We'll continue with an overview of post-training strategies. We'll discuss methods for supervised finetuning and learning from human feedback.
1/26 Week 4: LLM Inference We'll discuss advantages and trade-offs of various LM inference methods, in-context learning and test-time scaling.
1/28 Week 4: Evaluation & Benchmarking We'll discuss standards for evaluation of generative models. This includes common metrics for automatic evaluation (e.g. BLEU or Bertscore) as well as procedures for human evaluation.
2/2 Week 5: Guest Lecture
  • Mid-quarter final project report due by 11:59pm
2/4 Week 5: Mid-quarter Peer Review We will do peer review of mid-quarter final project reports. Each student should provide feedback on at least three reports other than their own.
2/9 Week 6: Model Interpretibility We'll discuss interpretibility of LLMs, uncertainty quantification and recent attempts to explain inner workings of black box models using mechanistic approaches. Students will walk through an in-class coding-intensive demo of modern interpretibility methods (laptops with Python required).
2/11 Week 6: Midterm
2/18 Week 7: Guest Lecture
  • Hw 3 is out (due 3/11 at 11:59pm)
2/23 Week 8: Knowledge Retrieval & QA We'll discuss knowledge retrieval methods, retrieval-augmented generation and question-answering.
2/25 Week 8: Conclusion We'll cover open challenges in NLP research and conclude with final remarks.
  • Final presentation slides due 3/1 at 11:59pm
3/2 Week 9: Final Presentations Schedule TBD
3/4 Week 9: Final Presentations Schedule TBD
3/9 Week 10: Final Presentations Schedule TBD
3/11 Week 10: Final Presentations Schedule TBD
  • Final project report due 3/16 by 11:59pm

Resources:

Assignments and announcements will be posted on Bruin Learn. We will be using Piazza for course discussion, including online help with homework assignments.

Grading:

Detailed guidelines for assignments will be released later in the quarter.

  • Homework assignments (45%)
    • Students will individually complete 3 written and coding problem sets related to weekly topics. The third homework assignment will cover material from the entire quarter, similar to a take-home final.
  • Course Project (30%)
    • Students will form groups of 4 and develop a NLP-based solution to a societal problem of the project team's choosing with comprehensive results and error analyses. Potential areas of interest are healthcare AI, legal NLP, content moderation, assistive technology for people with disabilities, urban planning or privacy and security. Please consult the professor if you are ensure a project idea is within scope.
    • This will be graded based on a 1-page project proposal (5%), a mid-project report (5%), final in-person presentations (5%) and a final write-up (15%).
  • In-Class Midterm Exam(15%)
    • Details TBD.
  • Participation (10%)
    • Students will be expected to attend class regularly and submit responses to in-class quizzes starting from week 2. Quizzes will be graded based only on completion. Students will also choose and present one NLP research paper in-person during the quarter. Students are expected to provide short, constructive feedback to their peers' mid-quarter reports and final presentations that can aid in editing final project write-ups.

Course Policies:

Late Policy. Students will have 5 late days that can be used on any homework assignment (or multiple homework assignments) without penalty. Since the final project is a group assignment there are no late days, but extensions will be considered under extraordinary circumstances. Students are expected to communicate potential presentation conflicts (e.g. illness, conference travel) to the instructor in advance.

Academic Honesty. Homework assignments are expected to be completed individually and the instructor will check for similarity in responses. For all assignments, any collaborators or other sources of help should be explicitly acknowledged. Violations of academic integrity (please consult the student conduct code) will be handled based on UCLA guidelines.

Accommodations. Our goal is to have a fair and welcoming learning environment. Students should contact the instructor at the beginning of the quarter if they will need special accomodations or have any concerns.

Use of ChatGPT and Other LLM Tools. Students are expected to first draft writing without any LLMs and all ideas presented must be their own. Students may use LLMs for grammer correction and minimal editing if they add an acknowledgement of this use. Any work suspected to be entirely AI-generated will be given a grade of 0.

Acknowledgements: The syllabus, assignments and lecture slides were adapted from Nanyun (Violet) Peng's course materials.