Mike Hostetler on ReqLLM

In this episode of the Elixir Mentor Podcast, I chat with Mike Hostetler, creator of the Jido agent framework and ReqLLM library. Mike shares his journey building a unified interface for calling multiple LLM providers in Elixir, addressing the frustrating inconsistencies between different AI APIs.

We dive into ReqLLM's architecture, exploring how it normalizes the differences between providers like OpenAI, Anthropic, Google, and more. Mike explains his decision to build a lightweight alternative to existing libraries like Instructor and LangChain, creating something that handles streaming, tool calling, and structured outputs with simple one-line function calls.

Mike demonstrates the library live, showing how to test 112+ models across different providers, handle streaming responses, and calculate token usage costs. We discuss the challenges of supporting multiple providers, from handling deprecated models to dealing with provider-specific headers and parameter variations.

The conversation also covers Jido's evolution, the upcoming Phoenix dashboard for managing agents, and Mike's vision for hierarchical agent systems in Elixir. We explore how ReqLLM fits into the broader Elixir AI ecosystem and discuss future plans for local LLM support and integration with frameworks like Ash.

Resources Mentioned:
- ReqLLM GitHub: https://github.com/agentjido/req_llm
- Jido Framework: https://agentjido.xyz
- models.dev: https://models.dev

Connect with Mike:
- GitHub: https://github.com/agentjido
- Website: https://mike-hostetler.com

SUPPORT ELIXIR MENTOR
- Elixir Mentor: https://elixirmentor.com