Spring 2024 MW 4:30pm-5:50pm, April 1st – June 5th
Location: Gates
B12
Edward
Chang (echang@cs.stanford.edu)
Adjunct Professor, Computer Science, Stanford
University
CA:
Matthew
Jin (mjin73@stanford.edu)
Office Hours: MW after class to 6:15pm at Gates B12 or
104.
Announcements
· (3/25/2024)
Textbook is online
at Kindle store.
· (3/07/2024)
Both SocraSynth and RAFEL are cutting edge research of LLMs. Reading materials are available on the sites,
and a course reader will be available by the start of the quarter.
· (1/29/2024) Course project can be on any subjects, e.g.,
psychology, laws, business, literature, political science, healthcare, computer
science, etc.
About
The 2024 edition of this course will delve
into Generative AI topics, encompassing innovations like SocraSynth, GPT-4,
ChatGPT, Gemini, and models of consciousness. In recent years, artificial
intelligence, particularly through deep learning, attention mechanisms, and
foundation models, has revolutionized technology. AI surpasses human
capabilities in various tasks, including computer vision and natural language
processing. Yet, we encounter challenges similar to
those from the initial AI boom five decades ago. This course will critically
analyze these challenges (such as issues with generalization, biases,
hallucinations, and reasoning) in prevalent AI algorithms like CNNs,
transformers, generative AI, and LLMs, including GPT and Gemini.
To overcome these hurdles, the course
will cover topics like transfer learning for data scarcity, knowledge-guided
multimodal learning for data diversity, and modeling of emotions, behaviors,
and ethics, along with multi-LLM collaborative dialogue.
Teaching will blend lectures with
project sessions. Guest speakers from academia and industry will share insights
on AI's niche applications, like in diagnosing and treating cancer or
depression. Students, from disciplines like CS, Business, Law, Medicine, and
Data Science, will undertake term projects that might include literature
reviews, idea development, and practical implementation. They are encouraged to
craft a project that complements their graduate research, fostering a deep,
integrative learning experience.
Note: The following
prerequisite for taking this course have been waved because of the ease-of-use
of ChatGPT.
Level: Senior/graduate of any majors.
Perquisite: Introductory course in
AI, Statistics, or Machine Learning.
Assignments
· No exams.
· Three assignments, using Gemini, ChatGPT and GPT-4 to
complete.
· Term project: A group project of two to three using LLMs, RAFEL,
and SocraSynth to address practical problems.
Textbook (required)
·
SocraSynth,
Edward Y. Chang, March, 2024
Grading
· Assignments 30%
· Class participation 10%
· Literature survey and project proposal 20%
· Project implementation and demo 40%
Syllabus (Tentative)
Date |
Description |
Course Materials |
Notes |
Week #1 4/1/2023 |
Course
Aims and Syllabus ChatGPT
Intro [slides]
|
Intro to AI, GAI and why you should or should not
take this course. |
E. Chang |
4/3/2023 |
LLMs
(Large Language Models) Part 1 of 5 Foundation
Models and ChatGPT [slides] |
Intro to GPT-4, ChatGPT, Llama and
Prompt engineering. [1] J. Wei, X. Wang, et al, Chain of thought prompting
elicits reasoning in large language models. Advances in Neural Information
Processing Systems, 2022. [link] [2] J. Jung, L. Qin, S. Welleck,
F. Brahman, C. Bhagavatula, R. L. Bras, and Y.
Choi. Maieutic prompting: Logically consistent reasoning with recursive
explanations. In Conference on Empirical Methods in Natural Language
Processing, 2022. [link] [3] Prompting
Large Language Models with the Socratic Method, E. Y. Chang, IEEE CCWC,
March 2023. [link] [4] Sparks of Artificial General Intelligence: Early
experiments with GPT-4, Sébastien Bubeck, Varun Chandrasekaran, et al, March 2003. [link] |
E. Chang Assignment #1: Design [your] chatbot functions,
using ChatGPT |
Week #2 4/8/2023 |
P4
Medicine, Part 1 of 3: History
of AI in Diagnosis [link] |
[1] Schwartz, W. B., R. S. Patil, and P. Szolovits. "AI in Medicine: where do we
stand." New England Journal of Medicine 316 (1987): 685-688.
[link]. [2] Wu, T. D. "Efficient Diagnosis of
Multiple Disorders Based on a Symptom Clustering Approach." Proceedings
of AAAI, 1990, pp. 357-364. [3] Universal Equivariant Multilayer Perceptrons, Siamak Ravanbakhsh, ICML, 2020. [4] Tricorder (medical IoTs), E. Y. Chang, et al.,
"Artificial Intelligence in XPRIZE DeepQ
Tricorder. " ACM MM Workshop for Personal Health and Health Care, 2017. [5] Szolovits,
P., and S. G. Pauker. "Categorical and
Probabilistic Reasoning in Medical Diagnosis.” Artificial
Intelligence 11(1-2), 1978: 115-144. [6] Patil, R. S., P. Szolovits, and W. B. Schwartz. "Causal
Understanding of Patient Illness in Medical Diagnosis."
In Proceedings of the Seventh International Joint Conference on
Artificial Intelligence. |
E. Chang |
4/10/2023 |
LLM,
Part 2 of 5: Prompt
template design principles [link] |
[1] Prompting Large Language Models with the
Socratic Method, E. Y. Chang, IEEE CCWC, March 2023. [link] [2] CRIT: Critical Reading Inquisitive Template,
E. Y. Chang. [link] [3] Noora.cs.stanford.edu |
E. Chang Assignment #1 due Assignment #2: Design and test prompting templates |
Week #3 4/15/2023 |
LLM,
Part 2 of 5 (finish the lecture) Prompt
template design principles +
project discussion |
[1] Prompting Large Language Models with the Socratic
Method, E. Y. Chang, IEEE CCWC, March 2023. [link] [2] CRIT: Critical Reading Inquisitive Template,
E. Y. Chang. [link] [3] Noora.cs.stanford.edu |
E. Chang |
4/17/2023 |
LLM,
Part 3 of 5: History
of NLP in one lecture. From
one-hot vector, word2vec to attention, transformers, BERT, and GPT [slides][video] |
[1] Attention is all you need, Ashish Vaswani, et
al., [link]. [2] BERT: Pre-training of Deep Bidirectional
Transformers for Language Understanding [link]. [3] Reformer: The Efficient Transformer [link]. [4] Perceiver: General Perception with Iterative
Attention [link]. |
E. Chang Assignment #2 due Assignment #3: Design chatbot states |
Week #4 4/22/2023 |
Psychiatric
Disorders, symptoms, and treatments [slides][video] |
Projections:
A Story of Human Emotions: Deisseroth, Karl |
Dr. Vikas Duvvuri, MD/PhD Stanford |
4/24/2023 |
LLM, Part
4 of 5 SocraSynth,
SocraHealth, SocraPlan, and SocraPedia. |
See relevant papers on www.socrasynth.com |
E. Chang |
Week #5 4/29/2023 |
P4
Medicine, Part 2 of 3: Healthcare
with Small Data [slides] Part#2
Training Your Own ChatGPT [slides]
|
[1] Daphne
Koller: Biomedicine and Machine
Learning, AI Podcast #93 with Lex Fridman, May
2020 [link] [2] Edward Y. Chang et al.,
"Context-Aware Symptom Checking for Disease Diagnosis Using
Hierarchical Reinforcement Learning." AAAI (2018). [3] Edward Y. Chang et al.,
"REFUEL: Exploring Sparse Features in Deep Reinforcement Learning for
Fast Disease Diagnosis." NIPS (2018). |
E. Chang |
5/1/2023 |
Project
proposal and related work presentation |
|
CA E. Chang |
Week #6 5/6/2023 |
P4
Medicine, Part 3 of 3: Frontier
Research [slides] (Healthcare
is already in the 3rd AI winter due to data and regulations.) |
[1] On Bottleneck of Graph Neural
Networks and its Practical Implications, Uri Alon, Eran Yahav, ICRL 2021. [2] Panel: VR/AR for Surgery and Medical Education,
April 2018 [link]. [3] Stanford SNI Talk: Advancing Healthcare w/ AI
and VR, Edward Y. Chang [link] (Stanford ID required). [4] The
problem of Protein
Folding, Wikipedia. |
E. Chang |
5/8/2023 |
LLM Part 5 of 5: RAFEL, Virtual Assistant and Augmented LM. [slides] |
[1] RAFEL, Edward Y. Chang,
March 2024 [3] Augmented Language Model,
a survey, Meta, 2023 [link]. |
E. Chang |
Week #7 5/13/2023 |
Consciousness
& Mind, Part 1 of 3: What is Consciousness? Rule of
Nature in Philosophy and Physics (lecture #14) [slides] |
[1] What is Life, Erwin Schrodinger, online book [link]. [2] COCOMO: Computational Consciousness Modeling,
E.Y. Chang, 2023 [link]. |
E. Chang |
5/15/2023 |
Consciousness
& Mind, Part 2 of 3: Computational Consciousness, COCOMO (lecture #15) [slides].
|
[1] Projections, by Karl
Deisseroth, Random House, 2021. [2] Discovery of a Perceptual
Distance Function for Measuring Image Similarity, Beitao
Li, Edward Chang, and Yi Wu, Multim. Syst., 2003
[link]. [3] COCOMO: Computational
Consciousness Modeling, E.Y. Chang, 2023 [link]. |
E. Chang |
Week #8 5/20/2023 |
Cancer,
Part 1 od 2: Cancer Causes and Diagnosis (lecture #12) |
[1] Principles and methods of integrative genomic
analyses in cancer, Vessela N. Kristensen, Ole
Christian Lingjærde, Hege G. Russnes,
Hans Kristian M. Vollan, et al., Nature Reviews,
2014. [2] Biomarker development in the precision medicine
era: lung cancer as a case study, Ashley J. Vargas, and Curtis C. Harris,
Natural Reviews, 2016. [optional] Artificial intelligence in radiology,
Ahmed Hosny, Chintan Parmar, John Quackenbush,
Lawrence H. Schwartz,and
Hugo J. W. L. Aerts, Nature Reviews, 2018. |
Dr. Melissa Ko, PhD, Cancer Biology, Stanford E. Chang |
5/22/2023 |
Cancer,
Part 2 of 2: Cancer Treatment and How AI May Help (lecture #13) [video][Slides] |
[1] Mass cytometry: blessed with the curse of
dimensionality, Evan W Newell & Yang Cheng, Nature Immunology, June
2016. [2] Next-Generation Machine Learning for Biological
Networks, Diogo M. Camacho, Katherine M. Collins,
Rani K. Powers, James C. Costello, and James J. Collins, Leading Edge
Review, June 2018. [optional] Personalized Cancer Models for Target,
Discovery and Precision Medicine, Carla Grandori1 and Christopher J. Kemp,
Trends in Cancer, CellPress Reviews, September
2018. |
Dr. Melissa Ko, PhD, Cancer Biology, Stanford |
Week #9 5/27/2023 |
Memorial
Holiday |
|
|
5/29/2023 |
Consciousness
& Mind, Part 3 of 3: GAI Alignment, Free Will, Ethics, and Mind; Course
Summary (lecture #16) |
[1] COCOMO: Computational Consciousness Modeling,
E.Y. Chang, 2023. |
E. Chang |
6/3/2023 |
Project
Presentation I |
Please sign up (sending us email) |
E. Chang |
Week #10 6/5 (last
day of class) |
Project
Presentations II |
Presentation time is flexible to arrange between
5/31 and 6/5. If later than 6/5,
then online presentation can be arranged. |
E. Chang CA |