API Testing & Validation: Catching Failures Before Production (with Aries)

A green CI check doesn't mean anybody validated anything. You can ship a service, watch every test pass, and still find out in production that nothing works. By then it's not a code review comment. It's an incident.

Validation is the cheapest place in your whole pipeline to catch that, and almost every team treats it as a checkbox at the end. We built Aries to fix it. In beta it took testing from weeks down to minutes.

Season 1 of AI Afterhours, Signals & Noise, is about how organizations cut through noise, find the signals that matter, and turn them into better decisions.

In Episode 6, API Assurance with Aries, I sat down with Varun Vemula, our CEO at AZTRA, to kick off the software and digital infrastructure arc. If you own APIs in production, you've lived this.

What we get into:

The support ticket that started Aries, and why the team who got paged couldn't fix it

Why every tool out there checks whether an API responds, and almost none check whether it actually did anything

Why two test cases can beat fifty, and how you know which ones to skip

How much of Aries is actually AI. Varun says 30%, and not for writing the tests

The customer in Amsterdam with an Ohio phone number, and why fake test data makes the whole run worthless

Why AI-assisted code made this harder, not easier

"You can generate two test cases which 100% covers everything, rather than generate 50 test cases that are of no value."

Varun Vemula, CEO, AZTRA

I looked up a Postman stat before we recorded. The average app has 28 to 50 APIs. That's a couple years old so it's probably low. And as endpoints grow, the ways they can contradict or fail grow even faster. It's multiplicative.

It's a bit ironic when you think about it. The whole point of wiring these endpoints together is to do things automatically. So why is the way we test them still somebody clicking buttons? There have never been more PRs or more incidents, and a lot of it is slop nobody checked. You build something, say it's done, then go to use it and nothing works, all the pages are broken. Validation is the long pole in the tent now. It's what stands between vibe coding and something that actually works on the other end.

Episode 7, Preventing API Failures at Scale, is next. We're staying in the Aries world, but going from catching problems in validation to keeping them from happening in the first place.

Listen to the full episode below. And if you want to try this on your own API, show us your spec and we'll tell you what we can discern from it and where the gaps are.

Thanks for tuning in. See you next episode.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit substack.aztra.ai