Carl Sable

Professor of Computer Engineering

ECE 467: Natural Language Processing

Catalog Data:

This course focuses on computer applications that involve the processing of written or spoken human languages. The exact content may vary from year to year. The course is divided into three parts. Topics from conventional, statistical natural language processing will likely include text normalization, N-grams, part-of-speech tagging, information retrieval, and text categorization. Topics from conventional computational linguistics will likely include grammars, parsing, and semantic representations. Topics from deep learning and NLP will likely include word embeddings, feed-forward neural networks, recurrent neural networks, sequence-to-sequence models, attention, and transformers. Course work will include programming projects and quizzes.

Topics:

  1. Introduction and course overview.
  2. Tokenization, words, and morphology.
  3. N-grams and conventional language models.
  4. Part-of-speech tagging.
  5. Vector-space models, information retrieval, and text categorization.
  6. Phrase-structure grammars and dependency grammars.
  7. Natural languages and psycholinguistics.
  8. Parsing.
  9. First-order logic and semantics.
  10. Feedforward neural networks.
  11. Word embeddings and neural language models.
  12. Recurrent neural networks and LSTMs.
  13. Encoder-decoder models, attention, and machine translation.
  14. Advanced topics: subword embeddings; question-answering; transformers; contextual embeddings and large language models (e.g., ELMo, BERT, GPT); Ethics and NLP

Course Outcomes:

Students will understand the fundamentals of conventional, statistical natural language processing (NLP); computational linguistics; and modern, deep learning approaches to NLP. Students will demonstrate their proficiency by implementing three programming projects, each requiring an in-depth understanding of one or more important NLP algorithms. Problem sets or tests will be used to evaluate various levels of knowledge of other NLP topics.

Students will attain:

  1. Knowledge of various subtopics of natural language processing, some in depth.
  2. Familiarity with many important NLP algorithms, methodologies, and resources.
  3. Experience developing large NLP projects.

Assessment Methods:

The three programming projects test in-depth understanding of important NLP algorithms. Quizzes, exams, or problem sets are used to evaluate various levels of knowledge of other NLP topics.