Interconnects Audio Nathan Lambert
-
- Technology
-
Audio format of posts on interconnects.ai -- generated with AI from the author.
-
RLHF: A thin line between useful and lobotomized
Many, many signs of life for preference fine-tuning beyond spoofing chat evaluation tools.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/how-rlhf-works-2
00:00 How RLHF works, part 2: A thin line between useful and lobotomized04:27 The chattiness paradox08:09 The mechanism for making models chattier10:42 Next steps for RLHF research
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/rlhf/img_012.webpFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/rlhf/img_018.pngFig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/rlhf/img_025.png -
Phi 3 and Arctic: Outlier LMs are hints
Models that seem totally out of scope from recent open LLMs give us a sneak peek of where the industry will be in 6 to 18 months.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/phi-3-and-arctic-llms
0:00 Phi 3 and Arctic: Outlier LMs are hints1:01 Arctic & open mixture of expert trends6:10 Phi 3, synthetic data, and small models
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/phi3/img_004.pngFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/phi3/img_008.pngFig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/phi3/img_018.png -
AGI is what you want it to be
Certain definitions of AGI are backing people into a pseudo-religious corner.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/agi-is-what-you-want-it-to-be
00:00 AGI is what you want it to be04:01 RL still rules the AGI discourse05:43 Modern AGI tests07:37 Agency and shifting goalposts
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/agi/img_018.pngFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/agi/img_020.png -
Llama 3: Scaling open LLMs to AGI
Meta shows that scaling won't be a limit for open LLM players in the near future.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/llama-3-and-scaling-open-llms
00:00 Llama 3; scaling open LLMs to AGI01:44 Pretraining, data, and basic evals06:06 Alignment and human evaluations10:08 Chatting with Meta AI and Llama 3 70B Instruct11:55 Same Llama license (mostly)12:52 The healthy open LLM ecosystem
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_011.jpegFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_013.pngFig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_015.pngFig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_020.pngFig 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_036.pngFig 6: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_040.pngFig 7: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_046.jpegFig 8: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_061.pngFig 9: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_063.webpFig 10: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_066.pngFig 11: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/llama3/img_068.jpeg -
Stop "reinventing" everything to "solve" alignment
Integrating some non computing science into reinforcement learning from human feedback can give us the models we want.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/reinventing-llm-alignment
0:00 Stop "reinventing" everything to "solve" AI alignment2:19 Social Choice for AI Alignment: Dealing with Diverse Human Feedback7:03 OLMo 1.7 7B: A truly open model with actually good benchmarks
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reinvention/img_013.pngFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reinvention/img_015.pngFig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reinvention/img_018.pngFig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reinvention/img_024.pngFig 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reinvention/img_027.png -
The end of the "best open LLM"
Modeling the compute versus performance tradeoff of many open LLMs.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/compute-efficient-open-llms
0:00 The end of the "best open LLM"3:05 Compute efficient open LLMs
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_004.jpegFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_009.pngFig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_014.pngFig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_016.pngFig 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_018.pngFig 6: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_020.pngFig 7: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_022.pngFig 8: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_024.pngFig 9: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/scaling/img_028.png