LLM Engineering - The Foundations
4.6
(13 ratings)
5 Weeks
·Cohort-based Course
LLM Engineering has expanded into our AI Engineering Bootcamp! Visit the link for more info! maven.com/aimakerspace/ai-eng-bootcamp
LLM Engineering - The Foundations
4.6
(13 ratings)
5 Weeks
·Cohort-based Course
LLM Engineering has expanded into our AI Engineering Bootcamp! Visit the link for more info! maven.com/aimakerspace/ai-eng-bootcamp
Course overview
Understand LLM training from first principles
Master Large Language Model architecture, pretraining, prompt engineering, fine-tuning, and alignment. From the Transformer to RLAIF.
Data science and engineering teams around the world are being asked to rapidly build LLM applications through prompt engineering and supervised fine-tuning. Some companies are even working to train their own proprietary domain-specific LLMs from scratch!
In order to build a "ChatGPT" for your customers or internal stakeholders, using your own proprietary data, you'll want to understand how GPT-style models are actually built, step-by-step.
From closed-source models like OpenAI's GPT-series, Google's PaLM models, Anthropic's Claude, others, to open-source models like LLaMA 2-70B, Mistral-7B, 01.AI's Yi-34B, Mosaic's MPT-30B, or Falcon-180B, these decoder-only architectures are at the core, made in the same way.
This course will provide you with the foundational concepts and code you need to demystify how these models are created, from soup to nuts and to actually get started training, fine-tuning, and aligning your own LLMs.
From there, it's up to you to make the business case, organize the data, and secure the compute to give your company and your career a competitive LLM advantage.
Who is this course for
01
Aspiring AI Engineers looking to explore new career opportunities in Generative AI
02
Data scientists and Machine Learning Engineers who want to train their own LLMs
03
Stakeholders interested in training and deploying proprietary LLMs and applications
What you’ll get out of this course
Understand Large Language Model transformer architectures and how they process text for next word prediction
Grok how base-model next word prediction is improved for chat models and aligned with humans through RL
Deep dive on quantization strategies for more efficiently leveraging and fine-tuning LLMs including LoRA/qLoRA
Learn the story of of OpenAI's Generative Pre-Trained Transformers models from GPT to GPT-4 Turbo
Understand how Meta's LLaMA 2 was trained and why it serves as a benchmark for open-source LLM developers
Train models yourself each class using the classic techniques of pretraining, fine-tuning, and reinforcement learning
Work with other talented builders to bring your very own custom GPT-style Large Language Model to life
Explore frontiers of LLM Engineering including Mixture of Experts (MOE) models and Small Language Models (SLMs)
This course includes
Interactive live sessions
Lifetime access to course materials
10 in-depth lessons
Direct access to instructor
14 projects to apply learnings
Guided feedback & reflection
Private community of peers
Course certificate upon completion
Maven Satisfaction Guarantee
This course is backed by Maven’s guarantee. You can receive a full refund within 14 days after the course ends, provided you meet the completion criteria in our refund policy.
LLM Engineering - The Foundations
Course syllabus
Week 1
Jan 9—Jan 14
Modules
- Session 1: The TransformerSession 1: The Transformer1 Lesson • 2 Projects
- Session 2: From GPT to GPT-2Session 2: From GPT to GPT-21 Lesson • 1 Project
Week 2
Jan 15—Jan 21
Modules
- Session 3: From GPT-2 to GPT-3Session 3: From GPT-2 to GPT-32 Lessons • 2 Projects
- Session 4: Quantization and PEFT-LoRASession 4: Quantization and PEFT-LoRA1 Lesson • 1 Project
Week 3
Jan 22—Jan 28
Modules
- Session 5: Alignment I: Prompt Engineering & Instruction TuningSession 5: Alignment I: Prompt Engineering & Instruction Tuning1 Lesson • 1 Project
- Session 6: Alignment II: Reinforcement Learning with Human Feedback (RLHF)Session 6: Alignment II: Reinforcement Learning with Human Feedback (RLHF)1 Lesson • 3 Projects
Week 4
Jan 29—Feb 4
Modules
- Session #7: Reinforcement Learning with AI Feedback (RLAIF)Session #7: Reinforcement Learning with AI Feedback (RLAIF)1 Lesson • 2 Projects
- Session #8: Frontiers of LLM EngineeringSession #8: Frontiers of LLM Engineering1 Lesson • 1 Project
Week 5
Feb 5—Feb 8
Modules
- Demo DayDemo Day1 Project
Post-Course
Modules
- From LLM Engineering to LLM OperationsFrom LLM Engineering to LLM Operations1 Lesson
4.6
(13 ratings)
What students are saying
Prerequisites
A strong background in fundamental Machine Learning and Deep Learning
Understanding supervised learning, unsupervised learning, and neural network architectures are required. Introductory NLP and CV knowledge is encouraged.
A ability to program in Python within a Jupyter Notebook Environment
Understand basic Python syntax and constructs. You should be comfortable training and evaluating simple ML & DL models using test, train, and dev sets.
📅 Detailed Schedule!
Understand how everything comes together in the course to provide a holistic overview of the how LLMs are engineered.
Get all the details about the assignments, associated papers, and key concepts you'll learn!
Send me the deets ✌️
Meet your instructors
Dr. Greg Loughnane
Founder & CEO @ AI Makerspace
I've worked as an AI product manager, university professor, data science consultant, AI startup advisor, and ML researcher; TEDx & keynote speaker, lecturing since 2013.
From 2021-2023 I worked at FourthBrain (Backed by Andrew Ng's AI Fund) to build industry-leading online bootcamps in ML Engineering and ML Operations (MLOps):
🔗 Resources links: Deeplearning.ai demos, AI Makerspace demos, LinkedIn, Twitter, YouTube, Blog.
Chris "The LLM Wizard 🪄" Alexiuk
Co-Founder & CTO @ AI Makerspace
I'm currently working as the Founding Machine Learning Engineer at Ox - but in my off time you can find me creating content for Machine Learning: either for the AI Makerspace, FourthBrain, or my YouTube Channel!
My motto is "Build, build, build!", and I'm excited to get building with all of you!
Be the first to know about upcoming cohorts
LLM Engineering - The Foundations
Bulk purchases
Course schedule
4-6 hours per week
Class!
Tuesdays & Thursdays, 4:00-6:00pm PT
- Jan 9th, 11th
- Jan 16th, 18th
- Jan 23rd, 25th
- Jan 30th, Feb 1st
Programming Assignments
2-4 hours per week
Each class period, we will get hands-on with Python coding homework!
Office Hours
Tuesdays and Fridays
- Greg Loughnane: Mondays @ 8 AM PT
- Chris Alexiuk: Fridays @ 3 PM PT
Build. Ship. Share.
Get hands-on with code, every class
We're here to teach concepts plus code. Never one or the other.
Pair programming made fun and easy
Grow your network. Build together. Feel the difference that expert facilitation makes.
Meet your accountability buddies
Join a community of doers, aimed in the same career direction as you are.
Frequently Asked Questions
What happens if I can’t make a live session?
I work full-time, what is the expected time commitment?
What’s the refund policy?
What if I'm not yet proficient in Python and the foundations of Machine Learning?
What if I don't know Git or how to use GitHub?
How can I learn more about AI Makerspace?
Are there any volume discounts if I want my whole team to take the course?
Be the first to know about upcoming cohorts
LLM Engineering - The Foundations
Bulk purchases