Training and Deploying Open-Source LLMs with Dr. Jon Krohn ODSC's Ai X Podcast

    • Technology

Listen on Apple Podcasts
Requires macOS 11.4 or higher

In this episode, we speak with Dr. Jon Krohn about the life cycle of open-source LLMs. Jon is a co-founder and chief data scientist at the machine learning company Nebula. He is the author of the book Deep Learning Illustrated, which was an instant #1 bestseller and was translated into seven languages. He is also the host of the fabulous SuperDataScience podcast, the data science industry’s most listened-to podcast. An incredible instructor and speaker, Jon’s workshops at ODSC conferences and other events are always one of our most popular.

Topics:
1. Guest Introduction
2. Definition of an open source LLMs and what it means to be truly open source
3. The importance of LLM weights and neural networks architecture for training
4. Transformer architecture
5. Apple expanding their AI team
6. What do I need to train or fine-tune an LLM
7. Key libraries for fine-tunning an LLM
8. The LoRA (Low-Rank Adaptation) technique for efficiently fine-tuning large language models
9. Testing and evaluating LLMs prior to deploying in production
10. Retrieval Augmented Generation (RAG)
11. Deploying LLM to production
12. How to keep inference costs down
13. How can people follow Jon’s content (see show notes also)
Show Notes:
More about Jon:
LinkedIn - https://www.linkedin.com/in/jonkrohn/
Jon’s YouTube Channel - https://www.youtube.com/c/jonkrohnlearns
Jon’s Monthly Newsletter - https://www.jonkrohn.com/

Tools and Resources:
Michael Nielsen's eBook on Neural Networks and Deep Learning - http://neuralnetworksanddeeplearning.com/
PyTorch Lightning is the deep learning framework - https://lightning.ai/docs/pytorch/stable/

Hugging Face Transformers Library - https://huggingface.co/docs/transformers/v4.17.0/en/index

Vicuna: An Open-Source Chatbot - https://lmsys.org/blog/2023-03-30-vicuna/

LoRA: Low-Rank Adaptation of Large Language Models - https://arxiv.org/abs/2106.09685

SDS 674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation) - https://www.superdatascience.com/podcast/parameter-efficient-fine-tuning-of-llms-using-lora-low-rank-adaptation

Unsloth for finetuning Llama 3, Mistral & Gemma - https://github.com/unslothai/unsloth

Phoenix: an open-source AI Observability & Evaluation tool - https://github.com/Arize-ai/phoenix

ODSC Podcast with Amber Roberts on Phoenix and troubleshooting LLMs - https://www.odsc.com/podcast/#e33

Weights & Biases - https://wandb.ai/site
This episode was sponsored by:
Ai+ Training https://aiplus.training/
Home to hundreds of hours of on-demand, self-paced AI training, ODSC interviews, free webinars, and certifications in in-demand skills like LLMs and Prompt Engineering

And created in partnership with ODSC https://odsc.com/
The Leading AI Training Conference, featuring expert-led, hands-on workshops, training sessions, and talks on cutting-edge AI topics and

Never miss an episode, subscribe now!

In this episode, we speak with Dr. Jon Krohn about the life cycle of open-source LLMs. Jon is a co-founder and chief data scientist at the machine learning company Nebula. He is the author of the book Deep Learning Illustrated, which was an instant #1 bestseller and was translated into seven languages. He is also the host of the fabulous SuperDataScience podcast, the data science industry’s most listened-to podcast. An incredible instructor and speaker, Jon’s workshops at ODSC conferences and other events are always one of our most popular.

Topics:
1. Guest Introduction
2. Definition of an open source LLMs and what it means to be truly open source
3. The importance of LLM weights and neural networks architecture for training
4. Transformer architecture
5. Apple expanding their AI team
6. What do I need to train or fine-tune an LLM
7. Key libraries for fine-tunning an LLM
8. The LoRA (Low-Rank Adaptation) technique for efficiently fine-tuning large language models
9. Testing and evaluating LLMs prior to deploying in production
10. Retrieval Augmented Generation (RAG)
11. Deploying LLM to production
12. How to keep inference costs down
13. How can people follow Jon’s content (see show notes also)
Show Notes:
More about Jon:
LinkedIn - https://www.linkedin.com/in/jonkrohn/
Jon’s YouTube Channel - https://www.youtube.com/c/jonkrohnlearns
Jon’s Monthly Newsletter - https://www.jonkrohn.com/

Tools and Resources:
Michael Nielsen's eBook on Neural Networks and Deep Learning - http://neuralnetworksanddeeplearning.com/
PyTorch Lightning is the deep learning framework - https://lightning.ai/docs/pytorch/stable/

Hugging Face Transformers Library - https://huggingface.co/docs/transformers/v4.17.0/en/index

Vicuna: An Open-Source Chatbot - https://lmsys.org/blog/2023-03-30-vicuna/

LoRA: Low-Rank Adaptation of Large Language Models - https://arxiv.org/abs/2106.09685

SDS 674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation) - https://www.superdatascience.com/podcast/parameter-efficient-fine-tuning-of-llms-using-lora-low-rank-adaptation

Unsloth for finetuning Llama 3, Mistral & Gemma - https://github.com/unslothai/unsloth

Phoenix: an open-source AI Observability & Evaluation tool - https://github.com/Arize-ai/phoenix

ODSC Podcast with Amber Roberts on Phoenix and troubleshooting LLMs - https://www.odsc.com/podcast/#e33

Weights & Biases - https://wandb.ai/site
This episode was sponsored by:
Ai+ Training https://aiplus.training/
Home to hundreds of hours of on-demand, self-paced AI training, ODSC interviews, free webinars, and certifications in in-demand skills like LLMs and Prompt Engineering

And created in partnership with ODSC https://odsc.com/
The Leading AI Training Conference, featuring expert-led, hands-on workshops, training sessions, and talks on cutting-edge AI topics and

Never miss an episode, subscribe now!

Top Podcasts In Technology

Acquired
Ben Gilbert and David Rosenthal
All-In with Chamath, Jason, Sacks & Friedberg
All-In Podcast, LLC
Lex Fridman Podcast
Lex Fridman
Search Engine
PJ Vogt, Audacy, Jigsaw
Hard Fork
The New York Times
TED Radio Hour
NPR