84 episodes

This CHAOSS Community podcast features members who spent considerable time and effort to understand open source community health and how we can measure it through metrics, analytics, and software. We invite guests to this podcast to talk about how they use open source community health metrics and software in their own open source communities, companies, or foundations. This podcast fills the gap with open source community metric definitions and software on one side and their use on the other side.

CHAOSScast CHAOSS Project

    • Technology

This CHAOSS Community podcast features members who spent considerable time and effort to understand open source community health and how we can measure it through metrics, analytics, and software. We invite guests to this podcast to talk about how they use open source community health metrics and software in their own open source communities, companies, or foundations. This podcast fills the gap with open source community metric definitions and software on one side and their use on the other side.

    Episode 84: Community Viability - how Verizon thinks about OSS risk

    Episode 84: Community Viability - how Verizon thinks about OSS risk

    Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!


    CHAOSScast – Episode 84


    In this episode of CHAOSScast, Dawn Foster, Matt Germonprez, Alice Sowerby, and guest Gary White, Principal Engineer at Verizon’s OSPO office, delve into the world of viability metrics models developed for assessing the risks associated with using open source software components. Gary explains the creation process of these models, their application within Verizon for software evaluation, and the significance of engaging with the open source community to enhance project viability. The conversations also explore the challenges and considerations in deploying these metrics within organizations, emphasizing the blend of policy enforcement and cultural influence to manage open source software dependencies effectively. Press download now to hear more!


    [00:02:30] Dawn asks Gary to elaborate on the choice of Verizon for the viability metrics models. He explains the creation of the first four metrics models for assessing risks in open source software components, and the development of a fifth model to simplify the original four. Also, he explains the importance of being quantitative about software library choices, influenced by a research paper from Carnegie Mellon and existing CHAOSS metrics.


    [00:05:16] Gary mentions using Augur for metrics collection at Verizon and the benefits of tracking with CHAOSS tools.


    [00:06:27] Matt asks Gary to provide an example of a metric used in the governance model, and he talks about the Libyears metric, which helps understand the total years behind all dependencies of a component, reflecting the risk associated with aging dependencies.


    [00:07:50] Alice wonders about the “happy region” for the Libyears metric and its implications on risk assessment.


    [00:09:25] Dawn asks Gary to discuss how these metrics are utilized at Verizon. He describes using these metrics to evaluate the viability of software at Verizon, including different use cases and dependency risks.


    [00:11:39] Alice explores how Gary considers the context in which components are used when calculating risk.


    [00:13:24] Matt asks about the process of engaging with the metrics models within the organization. Gary explains that the approach depends on several factors such as severity of finding, buy-in from the organization, and the organizational structure of the OSPO, and details the use of specific resources like the “endoflife.date.”


    [00:18:07] Gary outlines how Verizon integrates risk management frameworks with organizational tools like dashboards to disseminate collected data and foster buy-in for automated systems.
    [00:21:16] Alice asks Gary for advice on engaging with open source communities when viability metrics indicate potential issues. Gary highlights the importance of community and governance metrics in driving organizational support for critical open source projects.


    [00:22:43] Gary shares his experience in the CHAOSS group, emphasizing the value of diverse opinions in developing and validating viability metrics models.


    [00:24:33] Dawn highlights the significance of the discussions on viability and risk in the OSPO working group, emphasizing how these are critical concerns for OSPO leaders.


    [00:25:24] Dawn inquires about how Verizon uses CHAOSS metrics beyond viability assessment, particularly in open source management. Gary discusses leveraging CHAOSS metrics across various teams to judge component use and risk profiles and explains Verizon’s approach to using metrics involving both an educational component and a policy component.


    [00:27:33] Gary talks about focusing on the ongoing efforts to integrate and optimize the Augur system at Verizon, acknowledging Sean Goggins for his assistance, and expresses a desire to contribute back to the community, and exploring new metrics to trace and predict significant events in the open source ecosystem.


    Value Adds (Picks) of the

    • 34 min
    Episode 83: Metrics for Organizational and Digital Infrastructure with Edward Vielmetti

    Episode 83: Metrics for Organizational and Digital Infrastructure with Edward Vielmetti

    Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!


    CHAOSScast – Episode 83


    In this episode of CHAOSScast, Georg and Dawn chat with guest Edward Vielmetti, Developer Partner Manager at Equinix, where he oversees the Open Source Partner Program. Today, they delve into the significance of measuring open source community health using CHAOSS metrics. Edward discusses the importance of providing infrastructure support to open source projects and how Equinix uses CHAOSS metrics to evaluate project health and manage resources efficiently. The discussion also covers the challenges of maintaining open source project health, including governance, code quality, and resources, with insights into predictive metrics and the impact of corporate involvement in open source communities. Press download now to hear more!


    [00:01:36] Edward introduces himself, tells us what he does, provides a background on Equinix, and talks about their dedicated cloud offering and support for open source projects. He discusses the absence of formal CHAOSS metrics at Equinix but mentions they compare them with internal considerations to ensure project health.


    [00:06:24] Edward talks about external factors like internal conflicts or external shocks to the system and the importance of being a stabilizing force.


    [00:9:59] Georg outlines three categories of project health: community activity, code quality, and resources.


    [00:10:58] Edward talks about using spend as a top-line metric for resource adequacy and the importance of rapid build and test cycles for software projects.


    [00:15:33] Georg acknowledges Edward’s comprehensive view, noting the need for specialized infrastructure beyond what hosting platforms like GitHub and GitLab offer. Edward emphasizes that developing certain kinds of software requires direct access to hardware rather than virtualized environments.


    [00:19:06] Dawn brings the conversation back to CHAOSS, mentioning context working groups and Edward’s active participation in the corporate OSPO working group. Edward talks about the challenges at Equinix in forming a formal OSPO and the value of sharing and learning from peers through CHAOSS.


    [00:22:33] Dawn appreciated the diversity of companies in the CHAOSS OSPO working group and the broad exchange of ideas. Edward reflects on his long history with open source, noting the evolution and professionalization of the industry.


    [00:25:32] Georg asks about the future of open source and CHAOSS’s potential role, and Edward mentions the trend of open source projects changing control for financial gain and discusses how CHAOSS could help predict or quickly identify such changes. He proposes the collection of certain metrics, such as the number of legal notices a project receives, as indicators of the project’s environment.


    [00:29:44] Edward shares a story, without taking sides, about Terraform relicensing by HashiCorp and the subsequent forks of Terraform, focusing on the OpenTofu fork and the licensing issues around patching from differently licensed software.


    [00:34:05] Georg discusses observing early risk indicators in projects, such as when a single company’s influence increases, potentially raising the risk of unilateral changes, and he expresses a desire for a predictive model for open source project trajectories.


    [00:35:44] Dawn calls such predictive modeling difficult due to the rarity of events and stresses the importance of community participation for early detection of issues.


    [00:37:53] Georg brings up the Linkerd project’s approach to engaging with the vendor ecosystem and the changes in their release strategy to encourage commercial support, and Edward compares this with CentOS’s transition to CentOS Stream.


    [00:41:48] Georg reiterates the value of participation in open source to be aware of and potentially influence project developments.


    Value Adds (Picks) of the week:



    [00:42:29] Georg’s pick is

    • 45 min
    Episode 82: The AI Conundrum: Implications for OSPOs

    Episode 82: The AI Conundrum: Implications for OSPOs

    Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!


    CHAOSScast – Episode 82


    In this episode of CHAOSScast, host Dawn Foster brings together Matt Germonprez, Brian Proffitt, and Ashley Wolf to discuss the implications of Artificial Intelligence (AI) on Open Source Program Offices (OSPOs), including policy considerations, the potential for AI-driven contributions to create workload for maintainers, and the quality of contributions. They also touch on the use of AI internally within companies versus contributing back to the open source community, the importance of distinguishing between human and AI contributions, and the potential benefits and challenges AI introduces to open source project health and community metrics. The conversation strikes a balance between optimism for AI’s benefits and caution for its governance, leaving us to ponder the future of open source in an AI-integrated world. Press download to hear more!


    [00:03:20] The discussion begins on the role of OSPOs in AI policy making, and Ashley emphasizes the importance of OSPOs in providing guidance on generative AI tools usage and contributions within their organizations.


    [00:05:17] Brian observes a conservative reflex towards AI in OSPOs, noting issues around copyright, trust, and the status of AI as not truly open source.


    [00:07:10] Matt inquires about aligning different policies from various organizations, like GitHub and Red Hat, with those from the Linux Foundation and Apache Software Foundation regarding generative AI. Brian speaks about Red Hat’s approach to first figure out their policies before seeking alignment with others.


    [00:06:45] Ashley appreciates the publicly available AI policies from the Apache and Linux Foundations, noting that GitHub’s policies have been informed by long-term thinking and community feedback.


    [00:10:34] Dawn asks about potential internal conflict for GitHub employees given different AI policies at GitHub and other organizations like CNCF and Apache.


    [00:12:32] Ashley and Brian talk about what they see as the benefits of AI for OSPOs, and how AI can help scale OSPO support and act as a sounding board for new ideas.


    [00:15:32] Matt proposes a scenario where generative AI might increase individual contributions to high-profile projects like Kubernetes for personal gain, potentially burdening maintainers.


    [00:18:45] Dawn mentions Daniel Stenberg of cURL who has seen an influx of low-quality issues from AI models, Ashley points out the problem of “drive-by-contributions” and spam, particularly during events like Hacktoberfest, and emphasizes the role of OSPOs in education about responsible contributions, and Brian discusses potential issues with AI contributions leading to homogenization and the increased risk of widespread security vulnerabilities.


    [00:22:33] Matt raises another scenario questioning if companies might use generative AI internally as an alternative to open source for smaller issues without contributing back to the community. Ashley states 92% of developers are using AI code generation tools and cautions against creating code in a vacuum, and Brian talks about Red Hat’s approach.


    [00:27:18] Dawn discusses the impact of generative AI on companies that are primarily consumers of open source, rarely contributing back, questioning if they might start using AI to make changes instead of contributing. Brian suggests there might be a mixed impact and Ashley optimistically hopes the time saved using AI tools will be redirected to contribute back to open source.


    [00:29:49] Brian discusses the state of open source AI, highlighting the lack of a formal definition and ongoing efforts by the OSI and other groups to establish one, and recommends a fascinating article he read from Knowing Machines. Ashley emphasizes the importance of not misusing the term open source for AI until a formal definition is established.


    [00:32:42] Matt inquires how me

    • 39 min
    Episode 81: Managing Federal CHAOSS at CMS.gov

    Episode 81: Managing Federal CHAOSS at CMS.gov

    Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!


    CHAOSScast – Episode 80


    On today’s episode of CHAOSScast, we focus on the experiences and initiatives of the Open Source Program Office at the U.S. Centers for Medicare and Medicaid Services (CMS). Host Dawn Foster is joined by Sean Goggins along with guests, Remy DeCausemaker, Natalia Luzuriaga, Isaac Milarsky, and Aayat Ali, all from various backgrounds within the CMS, who share insights into their efforts in maintaining and promoting an open source culture within federal services. Key discussion points include the launch of the CMS’s first open source program office, the development of a maturity model framework to evaluate open source projects, the creation of tools such as Repo Scaffolder and Duplifier to support open source practices, and efforts towards open source software security. This episode emphasizes the distinct aspects of opens source work in government settings compared to the private sector and highlights upcoming presentations at conferences. Download this episode now to hear more!


    [00:02:21] Dawn asks about the team’s work at the U.S. Centers for Medicare and Medicaid Services. We start with Remy, who explains the launch of the first open source program office at a federal agency in the U.S. and details CMS’s mission to improve healthcare experience for over 150 million people and the role of the digital service within CMS.


    [00:05:36] Natalia discusses the maturity model framework developed to assess the open source maturity level of projects. She describes a “Repo Scaffolder” tool created in collaboration with the U.S. digital response to help projects align with the majority model, and she speaks about additional features for public repositories to aid in development.


    [00:10:51] Isaac takes over, explaining how they use Auger metrics and “Nadia labeling” to categorize projects and encourage the adoption of their maturity model. He details a metrics website that provides visual representations of project health and activity and introduces “Duplifier,” a deduplication tool for healthcare data, which uses an open source library called Splink.


    [00:15:14] Sean inquires how they actualize their user needs in metrics visualization and about the process that informs the creation of these visual metrics. Isaac addresses front-end design aspects of metric visualization and the importance of making the metrics understandable at a glance. Natalia emphasizing designing for both technical and non-technical stakeholders, ensuring metrics are clear and understandable.


    [00:17:44] Aayat discusses her role in strategy development and the creation of a CMS OSPO guide. She emphasizes advocacy withing CMS for open source and plans to conduct workshops and usability testing to determine which metrics are most valuable to stakeholders.


    [00:19:23] Remy talks about consulting with the chief information security officer and the chief information officer for internal metric priorities and engaging with an external OSPO metrics working group convened by CHAOSS for broader insights.


    [00:20:47] Dawn asks Remy for more details on the differences with government engagement in open source to the corporate environments. Remy describes the early journey of OSPOs at the federal level and contrasts it with his private sector experience.


    [00:25:18] Sean asks about what success would look like a year from now for the


    OSPO group’s work. Remy acknowledges the limited four-year term for digital service members, emphasizing the urgency to execute and make an impact within the next year. He highlights the transformative impact of Isaac and Natalia’s entrance into the program and the successful shipping of the metrics website, a deduplication tool, and other repositories.


    [00:27:50] Isaac envisions success as propagating maturity models and open source standards throughout the government, demonstrating value

    • 40 min
    Episode 80: Counting Potatoes vs. Computational Mysticism - Using CHAOSS for Research

    Episode 80: Counting Potatoes vs. Computational Mysticism - Using CHAOSS for Research

    Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!


    CHAOSScast – Episode 79


    In this episode, host Georg Link is joined by Daniel, Anita, Sophia, and Sean, to discuss their research experiences with CHAOSS metrics and software for open source community health analysis. They dive into various topics, such as collecting and interpreting data from different perspectives, considerations regarding privacy and ethics, and the importance of collaboration between academics and industry professionals. They also highlight some significant projects and studies where CHAOSS metrics and software were employed, and their hopes and concerns for the future direction of research in the field. Furthermore, they discuss the necessity of bridging the gap between academia and industry and touch on the importance of linguistics and cultural context when examining data. Download this episode now!


    [00:02:48] Anita discusses the history of open source software research and how CHAOSS provides a common framework for various metrics used by researchers, and Sean emphasizes the standardization of metrics by CHAOSS, which aids in consistency across research.


    [00:04:52] Sophia highlights the discrepancies in metric calculations and definitions, seeking standard methodologies, especially for non-academic publications, and Daniel reflects on the differences in research approaches between academia and industry, emphasizing the importance of methodological rigor.


    [00:08:25] Sean critiques academic papers for often lacking complete method descriptions, calling for a more rigorous methodological transparency, and Daniel shares about transitioning from academia to industry and the different expectations for communication and results.


    [00:10:44] Georg inquires about the impact of CHAOSS research capabilities, and Daniel explains that CHAOSS is shaping research by reflecting the interests and observations of its contributors.


    [00:12:16] Sean talks about the increased capacity for research offered by CHAOSS, particularly through tools like Grimoire Lab and Augur, Anita shares her experience using Grimoire Lab for creating interventions and dashboards for open source communities to monitor their projects, and Daniel adds historical context and mentions the importance of tools that allow the replication of analysis in research.


    [00:17:10] Georg introduces a study using CHAOSS metrics and software that hasn’t been officially published yet, and Sophia shares some details and explains the study’s premise.


    [00:21:00] Anita raises a philosophical point about the potential limitations of metrics, suggesting that they may only reflect what is observable and could lead to gamification if people optimize their behavior based on the metrics.


    [00:22:14] Sean speaks about the importance of deep field engagement and the combination of social science with data mining to fully understand the data’s underlying human behavior. Sophia shares her perspective from market research, discussing the design of surveys, the selection bias inherent in data collection, and the importance of understanding the population that is excluded by the research filters used.


    [00:25:56] Anita discusses the challenges of academic surveys, and Daniel discusses the bias that may arise from the data available.


    [00:28:10] Sophia contemplates the behavioral nuances dictated by different platforms’ processes, and Sean suggests a focus on common software engineering processes across different tools and advocates for social scientific research in open source to better understand the human aspects.


    [00:30:32] Georg transitions to discussing survey methodologies and their relation to CHAOSS metrics, and Anita shares her experiences with survey design for the international Apache Software Foundation community and implementation.


    [00:33:10] Daniel reflects on the collaborative effort with the ASF community to ensure the survey’s t

    • 52 min
    Episode 79: The Mechanics of CHAOSS: A Deep Dive into Open Source Community Health Analytics

    Episode 79: The Mechanics of CHAOSS: A Deep Dive into Open Source Community Health Analytics

    Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!


    In this collaboration between the CHAOSS Cast and Mechanical Ink podcasts, hosts Dawn Foster and Schalk Neethling are joined by guests Daniel Izquierdo and Sean Goggins to discuss open source community health metrics.


    The focus is on providing an overview of two projects under the CHAOSS (Community Health Analytics for Open Source Software) umbrella - GrimoireLab and Augur. These open source tools gather data from diverse sources and analyze it to provide insights into open-source community health.


    The episode brings together two fascinating open source projects - GrimoireLab and Augur - that aim to provide insights into the health of open-source software communities. By gathering data from various platforms and channels, and analyzing contributor activity, issues, pull requests, and conversations, these projects shine a light on the inner workings of open-source projects.


    You have Sean Goggins, a university researcher who co-created Augur to dissect community interactions at scale. Then you have Daniel Izquierdo whose startup Bitergia built GrimoireLab to offer open source analytics as a service. Both share insightful stories on the evolving landscape of inner source and metrics-driven community management. It's a great listen for anyone involved in running open-source projects or communities.


    Beyond project leads, the conversation also touches on why understanding community health is vital today for enterprise adopters to track their dependencies and influence. And as Dawn Foster from CHAOSS chips in, you get an important reminder to not get carried away chasing tools without clarity on what specific questions you want answered from the underlying data.


    Overall, a stimulating mix of history and future direction on using metrics to guide open-source communities toward greater sustainability.
    Guests



    Daniel Izquierdo: Co-founder and CEO of Bitergia, co-founder of CHAOSS, President of InnerSource Commons
    Sean Goggins: Professor of Computer Science at the University of Missouri, maintainer of Augur software package in CHAOSS


    Key Topics Discussed



    Origins and goals of the CHAOSS project
    Overview of GrimoireLab
    Overview of Augur
    Supporting analysis across diverse data sources
    Focus areas and roadmaps
    Getting involved in the projects


    Links Mentioned



    CHAOSS (chaoss.community)
    GrimoireLab (grimoirelab.github.io)
    Augur (augurlabs.io)
    Cauldron instance (cauldron.io)
    Public Augur instance (metrics.chaoss.io)
    OSS Compass (oss.compass.community)
    Special Guest: Schalk Neethling.
    Support CHAOSScast

    • 1 hr 1 min

Top Podcasts In Technology

Acquired
Ben Gilbert and David Rosenthal
Lex Fridman Podcast
Lex Fridman
All-In with Chamath, Jason, Sacks & Friedberg
All-In Podcast, LLC
Machines Like Us
The Globe and Mail
The TED AI Show
TED
Hard Fork
The New York Times