Adding Observability To Your Python Applications With OpenTelemetry

Summary

Once you release an application into production it can be difficult to understand all of the ways that it is interacting with the systems that it integrates with. The OpenTracing project and its accompanying ecosystem of technologies aims to make observability of your systems more accessible. In this episode Austin Parker and Alex Boten explain how the correlation of tracing and metrics collection improves visibility of how your software is behaving, how you can use the Python SDK to automatically instrument your applications, and their vision for the future of observability as the OpenTelemetry standard gains broader adoption.

Announcements

• Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
• When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
• Your host as usual is Tobias Macey and today I’m interviewing Austin Parker and Alex Boten about the OpenTelemetry project and its efforts to standardize the collection and analysis of observability data for your applications

Interview

• Introductions
• How did you get introduced to Python?
• Can you start by describing what OpenTelemetry is and some of the story behind it?
• How do you define observability and in what ways is it separate from the "traditional" approach to monitoring?
• What are the goals of the OpenTelemetry project?
• For someone who wants to begin using OpenTelemetry clients in their Python application, what is the process of integrating it into their application?
• How does the definition and adoption of a cross-language standard for telemetry data benefit the broader software community?
  • How do you avoid the trap of limiting the whole ecosystem to the lowest common denominator?
• What types of information are you focused on collecting and analyzing to gain insights into the behavior of applications and systems?
  • What are some of the challenges that are commonly faced in interpreting the collected data?
• With so many implementations of the specification, how are you addressing issues of feature parity?
• For the Python SDK, how is it implemented?
  • What are some of the initial designs or assumptions that have had to be revised or reconsidered as it gains adoption?
• What is your approach to integration with the broader ecosystem of tools and frameworks in the Python community?
• What are some of the interesting or unexpected challenges that you have faced or lessons that you have learned while working on instrumentation of Python projects?
• Once an application is instrumented, what are the options for delivering and storing the collected data?
• What are some of the most interesting, unexpected, or challenging lessons that you have learned while working on and with the OpenTelemetry ecosystem?
• What are some of the most interesting, innovative, or unexpected ways that you have seen components in the OpenTelemetry ecosystem used?
• When is OpenTelemetry the wrong choice?
• What is in store for the future of the OpenTelemetry project?

Keep In Touch

• Austin
  • @austinlparker on Twitter
  • austinlparker on GitHub
• Alex
  • LinkedIn
  • @codeboten on Twitter
  • codeboten on GitHub

Picks

• Tobias
  • Pulumi
    • Podcast Episode
• Austin
  • Helm 3
• Alex
  • Algorithms To Live By: The Computer Science Of Everyday Decisions by Brian Christian and Tom Griffiths

Closing Announcements

• Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers
• Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

• OpenTelemetry
• Lightstep
• OpenTracing
• OpenCensus
• Distributed Tracing
• Jaeger
• Zipkin
• Observability
• Kubernetes
• Spring
• Flask
• gRPC
• Structlog
• Filebeat
• W3C Trace Context
• OpenTelemetry Python SDK
• OpenTelemetry Django
• OpenTelemetry Flask
• OpenTelemetry Collector
• OTLP == Open Telemetry Protocol

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA