Personalized Plans#
With the scope and span of GenAI, there’s simultaneously:
a lot of topics, from fine-tuning, security, to safety
a lot of people with different backgrounds and interests
To guide someone, you have to start with where they are and understand what they want. That’s at least what my professional mountain guide, definitely not a nerd, and friend Max Lurie told me. On the right sidebar is a list of personas that correspond to where I see people start. Each line is a link that will take you to the topics that are most relevant to that group. Find the closest one to you and start learning.
Something missing? Let me know
Is a topic you’ve heard about not on this list? Open an issue ticket and let me know!
The App Developer and Finetuner#
Most people reading this website are in this group. These are developers looking to use GenAI tools or APIs in their work or product that they’re building; they don’t need to know the ground-level details of LLMs but just enough to get started. This is sequenced so you start with your end goal, the application you want to build, then work your way through the fundamentals with code.
Language Model Systems - Figure out what you’re going to build and how LLMs will fit in.
LLM Overview - Learn the low-level intuition of what happens.
Shakespeare LLM from Scratch - Learn how an LLM is trained hands-on with code.
NN Quickstart if new to ML/AI - Understand how mathematical models “program” themselves.
LLM APIs (Coming Soon)
Tokenizers (Coming soon)
Finetuning Overview - How to get an LLM to do what you want for your application.
Evaluation - How to check if it actually does the thing.
Applied Finetuning Guide (Coming soon)
Those Who Want the Fundamentals#
These are people who want to understand the core of Language Models, including their early history and all the details. While knowing Claude Shannon was the GOAT of language models might not help you build ChatGPT, it is a fascinating fact.
Small Language Fundamentals - The bare fundamentals of language models and their original implementations.
Neural Network Quickstart - Understand how mathematical models “program” themselves.
LLM Overview - Learn the low-level intuition of what happens.
Shakespeare LLM from Scratch - Learn how an LLM is trained hands-on with code.
Tokenizers (Coming Soon)
Finetuning Overview - How to get an LLM to do what you want for your application.
LLAMA 2 Deep Dive - The most open paper for training a modern LLM
Society and Security Specialists#
GenAI is changing society’s relationship with computers. The implications are broad-ranging across cybersecurity, economics, and policy. Folks in this category could be information security specialists all the way to policymakers.
Risk and Safety References - A general overview of the landscape with relevant papers and publications from a variety of perspectives
Research Scientist#
These are the people driving the state of the art forward. This is the frontier, so this is an area where the Guidebook will offer less help. However, if you’re looking to understand what these folks do, look at the section labeled Deep Dives on the left.
The Executive#
If you have no more than 2 hours, and just want the overview, these two videos from Andrej Karpathy will get you started. The first is a deep dive into training a toy model. The second is a Microsoft keynote that talks at a higher level but includes numerous details of how the production-grade systems are trained.
Andrej Karpathy’s GPT from Scratch#
Andrej takes a teacher-student approach here and shows how to train a smaller GPT on your computer. The dataset is interesting, but sufficiently small so you can train on your own computer. The codebase is also small and easy to read, and in two hours, you’ll see every piece of an end-to-end model in code.
Now, this won’t get you to a ChatGPT-style model, and he explains why in the video.
Andrej Karpathy’s State of GPT#
This second video is a conference presentation where Andrej explains all the parts it takes to make a production-grade chat model. Hearing it from the same person means the vocabulary aligns, making it simpler to correlate the learnings from the previous sessions with this one.