Codename Goose - Your Next Open Source AI Agent

Richard Moot: Hello and welcome to another episode of the Square Developer Podcast. I'm your host, Richard Moot, head of developer relations here at Square. And today I'm joined by my fellow developer relations engineer, Rizel, who's over working on Block Open Source. Hi Rizel, welcome to the podcast.

Rizel Scarlett: Hey, Richard. Thanks for having me. And I know it's so cool. We're like coworkers, but on different teams

Richard Moot: And you get to work on some of the, I'll admit I'm a little bit jealous. You get to work on some of the cool open source stuff, but I still get to poke around in there occasionally. But today we wanted to talk about one of our most recent releases is Goose, and I would like you to do the honors of, give us the quick pitch. What is Goose?

Rizel Scarlett: Goose is an on machine AI agent and it's open source. So when I say on machine, it's local. Unlike a lot of other AI tools that you use via the cloud, you have everything stored on your computer, private, you have control over the data, and you get to interact with different lms. You can choose whichever you want, whether it's GPT, sonnet, 3.5, whatever you prefer, you get to bring it.

Richard Moot: Awesome. And so I'm going to hopefully give a little bit more because I want to just kind of clarify for Square developers who might be coming in, they're like, they're just building other APIs, SDKs, trying to extend stuff for square sellers. So when we're talking about an agent, an agent, I always end up thinking the matrix, the agents and the matrix. And from what I understand, it's not too far off. You give it instructions and it will actually go and do things on your machine for you write two files, edit files, run commands. It's almost like doing things that a person could do on your computer for you.

Rizel Scarlett: Yes, exactly. That's a really good description. It doesn't just edit code for you. It can control your system. So I had it dimmed, the lights on my computer open different applications. You can really just automate anything even if you didn't know how to code.

Richard Moot: Yeah, I mean that's one of the things that I didn't even really think about when I first tried Goose. So one of the fun benefits of working here at Block is that I got to have fun with it before it actually went live. And one thing that I didn't really think about until I tried the desktop client and I forgot to allow the plug, there's two different ways you can interact with it. There's the CLI and the terminal, and then there's a desktop client, which I think right now works on Mac os.

Rizel Scarlett: Yes,

Richard Moot: I know there's big requests and to have it work in more than just Windows.

Rizel Scarlett: Yeah. Yeah. Right now, I mean we do have what I think is a working version of Windows, but the experience for the build time is not great. So we're still working through that.

Richard Moot: Yeah, well, having my own wrestling with working with the Windows sub Linux, I only really think of it as WSL. I've had so many headaches of trying to deal with networking and connecting and when do I need to switch to the power show versus a terminal, and it's all the reason I end up falling back to doing all of my development on my Mac.

Rizel Scarlett: Yeah. I haven't used the Windows computer since I was an IT support person. I don't even know what the new developments are now.

Richard Moot: Yeah, I mean I recently got burned by that where I didn't realize that in order to do certain virtualization stuff, you had to have a specific version of Windows, like some professional version, and then that enabled virtualization to run a VM of something interesting.

I think since then they've baked in the Windows sub Linux thing, which is basically just running Ubuntu in a virtualization for you. But that was an eyeopener, but thankfully Microsoft's working on fixing these things, but we digress. So coming back to Goose and what is it that most people have that you've sort of seen from the community as they've been starting to try it out and use Goose?

Rizel Scarlett: Yeah, I mean I just see people, well, a lot of it is mainly developers. That's the larger side of just using it to automate a lot of the tasks that they are doing. Maybe setting up, what am I trying to say, the boilerplate for their code or just sometimes other different things. I see people wanting to build local models and just in general or doing things with their kids, but I've also seen people doing silly experiments. This is where I find a lot of fun where people are having Goose talk to Goose or having a team of different, I guess geese, a team of agents and they're basically running a whole bunch of stuff. So they had one Goose be the PM and it was instructing all the engineer agents to perform different tasks. So it's a varied amount of things, but a lot of people are just trying to make their lives easier and have Goose do the mundane task in the background while they do the creative things. I've just been doing fun silly stuff. Like I had Goose play tic-tac-toe with me just for fun. I just wanted to see if it could do that and that was cool. Yeah.

Richard Moot: Have you beat it yet?

Rizel Scarlett: Every time I'm disappointed.

Richard Moot: You think it'd be way more advanced? I mean tech to can kind of, if I'm not mistaken, I think based on who goes first, it can be a determined game as long as you play with perfect strategy.

Rizel Scarlett: Yeah, I told it to play competitively. I'm still working on the perfect prompt. You always let me win Goose what's going on.

Richard Moot: Maybe that's part of the underlying LLMs is that they want to be helpful and so they think they're being helpful by letting you win, otherwise you wouldn't have fun.

Rizel Scarlett: That's true.

Richard Moot: Well, one of the things I was very fascinated by when first trying out the desktop client versus the CLI, because I habitually used the CLI version, but when I first opened up the desktop client, I had asked, what is it that you can do? And one of the things that it suggested that never even occurred to me was using Apple Scripts to run certain automations on your system. And I immediately just went, okay, can you organize my downloads folder and put everything? And it just immediately put everything in organized folders. And that's something I used to, I mean years ago, write my own quick little scripts to be like, oh, I need to move all these CSVs into someplace and PDFs. And it just immediately did it for me and it was just, that was amazing because now I can actually find where the certain things are.

Rizel Scarlett: That's so awesome. Yeah, I think you might've been using the computer controller extension, and that might be my favorite so far just because of, oh my gosh, it could actually, it's not just writing code for me. I'm like, okay, cool. There's other stuff that can do that Cursor does that as well, but it can tap into my computer system if I give it permission and move things around. I did a computer controller extension tutorial and I was just making it do silly stuff. Like I mentioned, it dimmed my computer screen. It opened up Safari and found classical music to play it, did some research on AI agents for me and put it in A CSV and then it turned back on the lights. It's so cool. I can just tell it, go do my own work for me and I'll it back.

Richard Moot: Yeah, that's great. And so you touched on something that I think is kind of an interesting part about it, and I feel like I want to come back to the part to really emphasize GOOSE is an open source project, and so it allows you to attach all of those various LLMs to sort of power the experience. But what you just touched on there is the extensions. So the way that it can do these things, could you tell us a little bit about what are extensions and how are they used by either Goose or the LLM? What is the relationship there?

Rizel Scarlett: Yeah, so extensions are basically, I guess you can think of it as extending it to different applications or different use cases. And we're doing that through a protocol called the Model Context Protocol, which Anthropic and us have been partnering on. And basically it allows any AI agent to kind of have access to different data. So for example, there's a Figma MCP or a model context protocol, and you can connect GOOSE to that MCP and tell it, Hey, here's some designs that I have, and Goose will be able to look at those and copy it rather than when you're maybe working with something like chat GBT, you have to go and give it context and be like, Hey, chat GBT, I'm working on this. Here's how this goes. And it takes up a lot of time. It'll just jump right in. And like you were saying, it's open source, so anybody can make MCP, you can connect it to any MCP out there that, I mean, some of them have to be honest, some CPS that are out there since it's open source, they don't all work, but the ones that do, you can connect it to Goose.

Richard Moot: Yeah. And so that's kind of like what you were originally talking about, the computer controller one.

Richard Moot: I'm going to hopefully describe this in a way that can make this visual for those that are listening in. But when you're using GOOSE in the terminal, when you first ever install it, it'll run you through a configuration of, Hey, it's basically setting up your profile and it says, which LLM do you want to connect to? And then you can kind of select from there and then it'll say, give me your credentials. And then after that you can get the option to, well actually maybe I'm jumping the gun here. I think it just gets you through storing that. And then you can have the option of once Goose is configured, you can toggle on certain extensions, extensions that are included, and then there's a pro