ThursdAI - The top AI news from the past week From Weights & Biases, Join AI Evangelist Alex Volkov and a panel of experts to cover everything important that happened in the world of AI from the past week

Hello hello everyone, this is Alex, typing these words from beautiful Seattle (really, it only rained once while I was here!) where I'm attending Microsoft biggest developer conference BUILD.
This week we saw OpenAI get in the news from multiple angles, none of them positive and Microsoft clapped back at Google from last week with tons of new AI product announcements (CoPilot vs Gemini) and a few new PCs with NPU (Neural Processing Chips) that run alongside CPU/GPU combo we're familiar with. Those NPUs allow for local AI to run on these devices, making them AI native devices!
While I'm here I also had the pleasure to participate in the original AI tinkerers thanks to my friend Joe Heitzberg who operates and runs the aitinkerer.org (of which we are a local branch in Denver) and it was amazing to see tons of folks who listen to ThursdAI + read the newsletter and talk about Weave and evaluations with all of them! (Btw, one the left is Vik from Moondream, which we covered multiple times). I
Ok let's get to the news:
TL;DR of all topics covered:
* Open Source LLMs
* HuggingFace commits 10M in ZeroGPU (X)
* Microsoft open sources Phi-3 mini, Phi-3 small (7B) Medium (14B) and vision models w/ 128K context (Blog, Demo)
* Mistral 7B 0.3 - Base + Instruct (HF)
* LMSys created a "hard prompts" category (X)
* Cohere for AI releases Aya 23 - 3 models, 101 languages, (X)
* Big CO LLMs + APIs
* Microsoft Build recap - New AI native PCs, Recall functionality, Copilot everywhere
* Will post a dedicated episode to this on Sunday
* OpenAI pauses GPT-4o Sky voice because Scarlet Johansson complained
* Microsoft AI PCs - Copilot+ PCs (Blog)
* Anthropic - Scaling Monosemanticity paper - about mapping the features of an LLM (X, Paper)
* Vision & Video
* OpenBNB - MiniCPM-Llama3-V 2.5 (X, HuggingFace)
* Voice & Audio
* OpenAI pauses Sky voice due to ScarJo hiring legal counsel
* Tools & Hardware
* Humane is looking to sell (blog)

Open Source LLMs
Microsoft open sources Phi-3 mini, Phi-3 small (7B) Medium (14B) and vision models w/ 128K context (Blog, Demo)
Just in time for Build, Microsoft has open sourced the rest of the Phi family of models, specifically the small (7B) and the Medium (14B) models on top of the mini one we just knew as Phi-3.
All the models have a small context version (4K and 8K) and a large that goes up to 128K (tho they recommend using the small if you don't need that whole context) and all can run on device super quick.
Those models have MIT license, so use them as you will, and are giving an incredible performance comparatively to their size on benchmarks. Phi-3 mini, received an interesting split in the vibes, it was really good for reasoning tasks, but not very creative in it's writing, so some folks dismissed it, but it's hard to dismiss these new releases, especially when the benchmarks are that great!
LMsys just updated their arena to include a hard prompts category (X) which select for complex, specific and knowledge based prompts and scores the models on those. Phi-3 mini actually gets a big boost in ELO ranking when filtered on hard prompts and beats GPT-3.5 😮 Can't wait to see how the small and medium versions perform on the arena.
Mistral gives us function calling in Mistral 0.3 update (HF)
Just in time for the Mistral hackathon in Paris, Mistral has released an update to the 7B model (and likely will update the MoE 8x7B and 8x22B Mixtrals) with function calling and a new vocab.
This is awesome all around because function calling is important for agenting capabilities, and it's about time all companies have it, and apparently the way Mistral has it built in matches the Cohere Command R way and is already supported in Ollama, using raw mode.
Big CO LLMs + APIs
Open AI is not having a good week - Sky voice has paused, Employees complain
OpenAI is in hot waters this week, starting with pausing the Sky voice (arguably the best most natural sounding voice out of the ones that launched) due to comp

Wow, holy s**t, insane, overwhelming, incredible, the future is here!, "still not there", there are many more words to describe this past week. (TL;DR at the end of the blogpost)
I had a feeling it's going to be a big week, and the companies did NOT disappoint, so this is going to be a very big newsletter as well.
As you may have read last week, I was very lucky to be in San Francisco the weekend before Google IO, to co-host a hackathon with Meta LLama-3 team, and it was a blast, I will add my notes on that in This weeks Buzz section.
Then on Monday, we all got to watch the crazy announcements from OpenAI, namely a new flagship model called GPT-4o (we were right, it previously was im-also-a-good-gpt2-chatbot) that's twice faster, 50% cheaper (in English, significantly more so in other languages, more on that later) and is Omni (that's the o) which means it is end to end trained with voice, vision, text on inputs, and can generate text, voice and images on the output.
A true MMIO (multimodal on inputs and outputs, that's not the official term) is here and it has some very very surprising capabilities that blew us all away. Namely the ability to ask the model to "talk faster" or "more sarcasm in your voice" or "sing like a pirate", though, we didn't yet get that functionality with the GPT-4o model, it is absolutely and incredibly exciting. Oh and it's available to everyone for free!
That's GPT-4 level intelligence, for free for everyone, without having to log in!
What's also exciting was how immediate it was, apparently not only the model itself is faster (unclear if it's due to newer GPUs or distillation or some other crazy advancements or all of the above) but that training an end to end omnimodel reduces the latency to incredibly immediate conversation partner, one that you can interrupt, ask to recover from a mistake, and it can hold a conversation very very well.
So well, that indeed it seemed like, the Waifu future (digital girlfriends/wives) is very close to some folks who would want it, while we didn't get to try it (we got GPT-4o but not the new voice mode as Sam confirmed) OpenAI released a bunch of videos of their employees chatting with Omni (that's my nickname, use it if you'd like) and many online highlighted how thirsty / flirty it sounded. I downloaded all the videos for an X thread and I named one girlfriend.mp4, and well, just judge for yourself why:
Ok, that's not all that OpenAI updated or shipped, they also updated the Tokenizer which is incredible news to folks all around, specifically, the rest of the world. The new tokenizer reduces the previous "foreign language tax" by a LOT, making the model way way cheaper for the rest of the world as well
One last announcement from OpenAI was the desktop app experience, and this one, I actually got to use a bit, and it's incredible. MacOS only for now, this app comes with a launcher shortcut (kind of like RayCast) that let's you talk to ChatGPT right then and there, without opening a new tab, without additional interruptions, and it even can understand what you see on the screen, help you understand code, or jokes or look up information. Here's just one example I just had over at X. And sure, you could always do this with another tab, but the ability to do it without context switch is a huge win.
OpenAI had to do their demo 1 day before GoogleIO, but even during the excitement about GoogleIO, they had announced that Ilya is not only alive, but is also departing from OpenAI, which was followed by an announcement from Jan Leike (who co-headed the superailgnment team together with Ilya) that he left as well. This to me seemed like a well executed timing to give dampen the Google news a bit.
Google is BACK, backer than ever, Alex's Google IO recap
On Tuesday morning I showed up to Shoreline theater in Mountain View, together with creators/influencers delegation as we all watch the incredible firehouse of announcements that Google has prepared for us.
TL;DR -

Hey 👋 (show notes and links a bit below)
This week has been a great AI week, however, it does feel like a bit "quiet before the storm" with Google I/O on Tuesday next week (which I'll be covering from the ground in Shoreline!) and rumors that OpenAI is not just going to let Google have all the spotlight!
Early this week, we got 2 new models on LMsys, im-a-good-gpt2-chatbot and im-also-a-good-gpt2-chatbot, and we've now confirmed that they are from OpenAI, and folks have been testing them with logic puzzles, role play and have been saying great things, so maybe that's what we'll get from OpenAI soon?
Also on the show today, we had a BUNCH of guests, and as you know, I love chatting with the folks who make the news, so we've been honored to host Xingyao Wang and Graham Neubig core maintainers of Open Devin (which just broke SOTA on Swe-Bench this week!) and then we had friends of the pod Tanishq Abraham and Parmita Mishra dive deep into AlphaFold 3 from Google (both are medical / bio experts).
Also this week, OpenUI from Chris Van Pelt (Co-founder & CIO at Weights & Biases) has been blowing up, taking #1 Github trending spot, and I had the pleasure to invite Chris and chat about it on the show!
Let's delve into this (yes, this is I, Alex the human, using Delve as a joke, don't get triggered 😉)
TL;DR of all topics covered (trying something new, my Raw notes with all the links and bulletpoints are at the end of the newsletter)
* Open Source LLMs
* OpenDevin getting SOTA on Swe-Bench with 21% (X, Blog)
* DeepSeek V2 - 236B (21B Active) MoE (X, Try It)
* Weights & Biases OpenUI blows over 11K stars (X, Github, Try It)
* LLama-3 120B Chonker Merge from Maxime Labonne (X, HF)
* Alignment Lab open sources Buzz - 31M rows training dataset (X, HF)
* xLSTM - new transformer alternative (X, Paper, Critique)
* Benchmarks & Eval updates
* LLama-3 still in 6th place (LMsys analysis)
* Reka Core gets awesome 7th place and Qwen-Max breaks top 10 (X)
* No upsets in LLM leaderboard
* Big CO LLMs + APIs
* Google DeepMind announces AlphaFold-3 (Paper, Announcement)
* OpenAI publishes their Model Spec (Spec)
* OpenAI tests 2 models on LMsys (im-also-a-good-gpt2-chatbot & im-a-good-gpt2-chatbot)
* OpenAI joins Coalition for Content Provenance and Authenticity (Blog)
* Voice & Audio
* Udio adds in-painting - change parts of songs (X)
* 11Labs joins the AI Audio race (X)
* AI Art & Diffusion & 3D
* ByteDance PuLID - new high quality ID customization (Demo, Github, Paper)
* Tools & Hardware
* Went to the Museum with Rabbit R1 (My Thread)
* Co-Hosts and Guests
* Graham Neubig (@gneubig) & Xingyao Wang (@xingyaow_) from Open Devin
* Chris Van Pelt (@vanpelt) from Weights & Biases
* Nisten Tahiraj (@nisten) - Cohost
* Tanishq Abraham (@iScienceLuvr)
* Parmita Mishra (@prmshra)
* Wolfram Ravenwolf (@WolframRvnwlf)
* Ryan Carson (@ryancarson)
Open Source LLMs
Open Devin getting a whopping 21% on SWE-Bench (X, Blog)
Open Devin started as a tweet from our friend Junyang Lin (on the Qwen team at Alibaba) to get an open source alternative to the very popular Devin code agent from Cognition Lab (recently valued at $2B 🤯) and 8 weeks later, with tons of open source contributions, >100 contributors, they have almost 25K stars on Github, and now claim a State of the Art score on the very hard Swe-Bench Lite benchmark beating Devin and Swe-Agent (with 18%)
They have done so by using the CodeAct framework developed by Xingyao, and it's honestly incredible to see how an open source can catch up and beat a very well funded AI lab, within 8 weeks! Kudos to the OpenDevin folks for the organization, and amazing results!
DeepSeek v2 - huge MoE with 236B (21B active) parameters (X, Try It)
The folks at DeepSeek is releasing this huge MoE (the biggest we've seen in terms of experts) with 160 experts, and 6 experts activated per forward pass. A similar trend from the Snowflake team, just extended even longer. They also introduce a lot of technical details and

Hey 👋 Look it May or May not be the first AI newsletter you get in May, but it's for sure going to be a very information dense one. As we had an amazing conversation on the live recording today, over 1K folks joined to listen to the first May updates from ThursdAI.
As you May know by now, I just love giving the stage to folks who are the creators of the actual news I get to cover from week to week, and this week, we had again, 2 of those conversations.
First we chatted with Piotr Padlewski from Reka, the author on the new Vibe-Eval paper & Dataset which they published this week. We've had Yi and Max from Reka on the show before, but it was Piotr's first time and he was super super knowledgeable, and was really fun to chat with.
Specifically, as we at Weights & Biases launch a new product called Weave (which you should check out at https://wandb.me/weave) I'm getting more a LOT more interested in Evaluations and LLM scoring, and in fact, we started the whole show today with a full segment on Evals, Vibe checks and covered a new paper from Scale about overfitting.
The second deep dive was with my friend Idan Gazit, from GithubNext, about the new iteration of Github Copilot, called Copilot Workspace. It was a great one, and you should definitely give that one a listen as well

TL;DR of all topics covered + show notes
* Scores and Evals
* No notable changes, LLama-3 is still #6 on LMsys
* gpt2-chat came and went (in depth chan writeup)
* Scale checked for Data Contamination on GSM8K using GSM-1K (Announcement, Paper)
* Vibes-Eval from Reka - a set of multimodal evals (Announcement, Paper, HF dataset)
* Open Source LLMs
* Gradient releases 1M context window LLama-3 finetune (X)
* MaziyarPanahi/Llama-3-70B-Instruct-DPO-v0.4 (X, HF)
* Nous Research - Hermes Pro 2 - LLama 3 8B (X, HF)
* AI Town is running on Macs thanks to Pinokio (X)
* LMStudio releases their CLI - LMS (X, Github)
* Big CO LLMs + APIs
* Github releases Copilot Workspace (Announcement)
* AI21 - releases Jamba Instruct w/ 256K context (Announcement)
* Google shows Med-Gemini with some great results (Announcement)
* Claude releases IOS app and Team accounts (X)
* This weeks Buzz
* We're heading to SF to sponsor the biggest LLama-3 hackathon ever with Cerebral Valley (X)
* Check out my video for Weave our new product, it's just 3 minutes (Youtube)
* Vision & Video
* Intern LM open sourced a bunch of LLama-3 and Phi based VLMs (HUB)
* And they are MLXd by the "The Bloke" of MLX, Prince Canuma (X)
* AI Art & Diffusion & 3D
* ByteDance releases Hyper-SD - Stable Diffusion in a single inference step (Demo)
* Tools & Hardware
* Still haven't open the AI Pin, and Rabbit R1 just arrived, will open later today
* Co-Hosts and Guests
* Piotr Padlewski (@PiotrPadlewski) from Reka AI
* Idan Gazit (@idangazit) from Github Next
* Wing Lian (@winglian)
* Nisten Tahiraj (@nisten)
* Yam Peleg (@yampeleg)
* LDJ (@ldjconfirmed)
* Wolfram Ravenwolf (@WolframRvnwlf)
* Ryan Carson (@ryancarson)
Scores and Evaluations
New corner in today's pod and newsletter given the focus this week on new models and comparing them to existing models.
What is GPT2-chat and who put it on LMSys? (and how do we even know it's good?)
For a very brief period this week, a new mysterious model appeared on LMSys, and was called gpt2-chat. It only appeared on the Arena, and did not show up on the leaderboard, and yet, tons of sleuths from 4chan to reddit to X started trying to figure out what this model was and wasn't.
Folks started analyzing the tokenizer, the output schema, tried to get the system prompt and gauge the context length. Many folks were hoping that this is an early example of GPT4.5 or something else entirely.
It did NOT help that uncle SAMA first posted the first tweet and then edited it to remove the - and it was unclear if he's trolling again or foreshadowing a completely new release or an old GPT-2 but retrained on newer data or something.
The model was really surprisingly good, s

Hey hey folks, happy ThursdAI 🎉
Not a lot of house-keeping here, just a reminder that if you're listening or reading from Europe, our European fullyconnected.com conference is happening in May 15 in London, and you're more than welcome to join us there. I will have quite a few event updates in the upcoming show as well.
Besides this, this week has been a very exciting one for smaller models, as Microsoft teased and than released Phi-3 with MIT license, a tiny model that can run on most macs with just 3.8B parameters, and is really punching above it's weights. To a surprising and even eyebrow raising degree! Let's get into it 👇
ThursdAI - Recaps of the most high signal AI weekly spaces is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

TL;DR of all topics covered:
* Open Source LLMs
* Microsoft open sources Phi-3 (X, HF)
* LLama3 70B top5 (no top 6) on LMsys (LMsys Arena)
* Snowflake open sources Arctic - A massive hybrid MoE (X, Try it, HF)
* Evolutionary Model merges support in MergeKit (Blog)
* Llama-3 8B finetunes roundup - Longer Context (128K) and Dolphin & Bagel Finetunes
* HuggingFace FINEWEB - a massive 45TB (the GPT4 of datasets) and 15T tokens high quality web data dataset (HF)
* Cohere open sourced their chat interface (X)
* Apple open sources OpenElm 4 models + training library called corenet (HF, Github, Paper)
* Big CO LLMs + APIs
* Google Gemini 1.5 pro is #2 on LMsys arena
* Devin is now worth 2BN and Perplexity is also a Unicorn
* A new comer called Augment (backed by Eric Schmidt) is now coming out of stealth (X)
* Vision & Video
* Adobe releases VideoGigaGAN - high quality upscaler with temporal consistency (paper)
* TLDraw autocomplete UI demo (X)
* This Weeks Buzz - What I learned in WandB this week
* Joe Spisak talk about Llama3 on Stage at WandB Fully connected (Full Talk, TLDR)
* Voice & Audio
* Play.ai (previously play.ht) releases conversational Voice AI platform (X)
* AI Art & Diffusion & 3D
* IMGsys.org- like LMsys but for image generation model + leaderboard from FAL (try it)
* Tools & Hardware
* Rabbit R1 release party & no shipping update in sight
* I'm disillusioned about my AI Pin and will return it
Open Source LLMs
Llama-3 1 week-aversary 🎂 - Leaderboard ranking + finetunes
Well, it's exactly 1 week since we got Llama-3 from Meta and as expected, the rankings show a very very good story. (also it was downloaded over 1.2M times and already has 600 derivatives on HuggingFace)
Just on Monday, Llama-3 70B (the bigger version) took the incredible 5th place (now down to 6th) on LMSys, and more surprising, given that the Arena now has category filters (you can filter by English only, Longer chats, Coding etc) if you switch to English Only, this model shows up 2nd and was number 1 for a brief period of time.
So just to sum up, an open weights model that you can run on most current consumer hardware is taking over GPT-4-04-94, Claude Opus etc'
This seems dubious, because well, while it's amazing, it's clearly not at the level of Opus/Latest GPT-4 if you've used it, in fact it fails some basic logic questions in my tests, but it's a good reminder that it's really hard to know which model outperforms which and that the arena ALSO has a bias, of which people are using it for example and that evals are not a perfect way to explain which models are better.
However, LMsys is a big component of the overall vibes based eval in our community and Llama-3 is definitely a significant drop and it's really really good (even the smaller one)
One not so surprising thing about it, is that the Instruct version is also really really good, so much so, that the first finetunes of Eric Hartfords Dolphin (Dolphin-2.8-LLama3-70B) is improving just a little bit over Meta's own instruct version, which is done very well.
Per Joe Spisak (Program Manager @ Meta AI) chat at the Weights & Biases conference last week (which you

Happy LLama 3 day folks! After a lot of rumors, speculations, and apparently pressure from the big Zuck himself, we finally can call April 18th, 2024, LLaMa 3 day!
I am writing this, from a lobby of the Mariott hotel in SF, where our annual conference is happening called Fully Connected, and I recorded today's episode from my hotel room. I really wanna shout out how awesome it was to meet folks who are listeners of the ThursdAI pod and newsletter subscribers, participate in the events, and give high fives.
During our conference, we had the pleasure to have Joe Spisak, the Product Director of LLaMa at Meta, to actually announce LLaMa3 on stage! It was so exhilarating, I was sitting in the front row, and then had a good chat with Joe outside of the show 🙌
The first part of the show was of course, LLaMa 3 focused, we had such a great time chatting about the amazing new 8B and 70B models we got, and salivating after the announced but not yet released 400B model of LLaMa 3 😮
We also covered a BUNCH of other news from this week, that was already packed with tons of releases, AI news and I was happy to share my experiences running a workshop a day before our conference, with focus on LLM evaluations. (If there's an interest, I can share my notebooks and maybe even record a video walkthrough, let me know in the comments)
Ok let's dive in 👇
Happy LLama 3 day 🔥
The technical details
Meta has finally given us what we're all waiting for, an incredibly expensive (2 clusters of 24K H100s over 15 Trillion tokens) open weights models, the smaller 8B one and the larger 70B one.
We got both instruction fine tune and base models, which are great for finetuners, and worth mentioning that this is a dense model (not a mixture of experts, all the parameters are accessible for the model during inference)
It is REALLY good at benchmarks, with the 7B model beating the previous (LLaMa 2 70B) on pretty much all benchmarks, and the new 70B is inching on the bigger releases from the past month or two, like Claude Haiku and even Sonnet!
The only downsides are the 8K context window + non multimodality, but both are coming according to Joe Spisak who announced LLama3 on stage at our show Fully Connected 🔥
I was sitting in the front row and was very excited to ask him questions later!
By the way, Joe did go into details they haven't yet talked about pulblicly (see? I told you to come to our conference! and some of you did!) and I've been live-tweeting his whole talk + the chat outside with the "extra" spicy questions and Joes winks haha, you can read that thread here
The additional info
Meta has also partnered with both Google and Bing (take that OpenAI) and inserted LLama 3 into the search boxes of Facebook, Instagram, Messenger and Whatsapp plus deployed it to a new product called meta.ai (you can try it there now) and is now serving LLama 3 to more than 4 Billion people across all of those apps, talk about compute cost!
Llama 3 also has a new Tokenizer (that Joe encouraged us to "not sleep on") and a bunch of new security tools like Purple LLama and LLama Guard. PyTorch team recently released finetuning library called TorchTune is now supporting LLama3 finetuning natively out of the box as well (and integrates Wandb as it's first party experiment tracking tool)
If you'd like more details, directly from Joe, I was live tweeting his whole talk, and am working at getting the slides from our team. We'll likely have a recording as well, will post it as soon as we have it.
Here's a TL;DR (with my notes for the first time) of everything else we talked about, but given today is LLaMa day, and I still have to do fully connected demos, I will "open source" my notes and refer you to the podcast episode to hear more detail about everything else that happened today 🫡
TL;DR of all topics covered:
* Meta releases LLama 3 -8B, 70B and later 400B (Announcement, Models, Try it, Run Locally)
* Open Source LLMs
* Meta LLama 3