Software Engineering Institute (SEI) Podcast Series Members of Technical Staff at the Software Engineering Institute

At the request of the White House, the Office of the Director of National Intelligence (ODNI) began exploring use cases for large language models (LLMs) within the Intelligence Community (IC). As part of this effort, ODNI sponsored the Mayflower Project at Carnegie Mellon University’s Software Engineering Institute (SEI) from May 2023 through September 2023. The Mayflower Project attempted to answer the following questions:
How might the IC set up a baseline, stand-alone LLM? How might the IC customize LLMs for specific intelligence use cases? How might the IC evaluate the trustworthiness of LLMs across use cases? In this SEI Podcast, Shannon Gallagher, AI engineering team lead, and Rachel Dzombak, special advisor to the director of the SEI’s AI Division, discuss the findings and recommendations from the Mayflower Project and provides additional background information about LLMs and how they can be engineered for national security use cases.

Modern software engineering practices of Agile and DevSecOps have provided a foundation for producing working software products faster and more reliably than ever before. Far too often, however, these practices do not address the non-software concerns of business mission and capability delivery even though these concerns are critical to the successful delivery of a software product. Through our work with government organizations, we have found that expanding DevSecOps beyond product development enables other teams to increase their capabilities and improve their processes. Agile methodologies are also being used for complex system and hardware developments. In this podcast from the Carnegie Mellon University Software Engineering Institute, Lyndsi Hughes, a senior systems engineer and David Sweeney, an associate software developer, both with the SEI CERT Division, share their experiences leveraging DevSecOps pipelines in atypical situations in support of teams focused on the capability delivery and business mission for their organizations.

Increasingly in government acquisition of software-intensive systems, we are seeing programs using Agile development methodology and earned value management. While there are many benefits to using both Agile and EVM, there are important considerations that software program managers must first address. In this podcast, Patrick Place, a senior engineer, and Stephen Wilson, a test engineer, both with the SEI Agile Transformation Team, discuss seven considerations for successful use of Agile and EVM.
 

As developers continue to build greater autonomy into cyber-physical systems (CPSs), such as unmanned aerial vehicles (UAVs) and automobiles, these systems aggregate data from an increasing number of sensors. However, more sensors not only create more data and more precise data, but they require a complex architecture to correctly transfer and process multiple data streams. This increase in complexity comes with additional challenges for functional verification and validation, a greater potential for faults, and a larger attack surface. What’s more, CPSs often cannot distinguish faults from attacks. To address these challenges, researchers from the SEI and Georgia Tech collaborated on an effort to map the problem space and develop proposals for solving the challenges of increasing sensor data in CPSs. In this podcast from the Carnegie Mellon University Software Engineering Institute, Jerome Hugues, a principal researcher in the SEI Software Solutions Division, discusses this collaboration and its larger body of work, Safety Analysis and Fault Detection Isolation and Recovery (SAFIR) Synthesis for Time-Sensitive Cyber-Physical Systems.

To better understand the potential uses of large language models (LLMs) and their impact, a team of researchers at the Carnegie Mellon University Software Engineering Institute CERT Division conducted four in-depth case studies. The case studies span multiple domains and call for vastly different capabilities. In this podcast, Matthew Walsh, a senior data scientist in CERT, and Dominic Ross, Multi-Media Design Team lead, discuss their work in developing the four case studies as well as limitations and future uses of ChatGPT.

Research and development of quantum computers continues to grow at a rapid pace. The U.S. government alone spent more than $800 million on quantum information science research in 2022. Thomas Scanlon, who leads the data science group in the SEI CERT Division, was recently invited to be a participant in the Workshop on Cybersecurity of Quantum Computing, co-sponsored by the National Science Foundation (NSF) and the White House Office of Science and Technology Policy, to examine the emerging field of cybersecurity for quantum computing. In this podcast from the Carnegie Mellon University Software Engineering Institute, Scanlon discusses how to create the discipline of cyber protection of quantum computing and outlines six areas of future research in quantum cybersecurity.