Data poisoning—where adversaries tamper with training data to corrupt model behavior—poses significant risks as AI adoption expands across critical sectors. Organizations without mechanisms in place to detect or prevent data poisoning are open to an avenue of attack that, once exploited, is difficult to remediate. Machine unlearning and model retraining are not always viable or effective solutions. In today's operational climate, where threat actors look to influence models and degrade the trust of users through incorrect behaviors, preventing data poisoning is more important than ever.
In this episode of the SEI Podcast Series, Julie Lawler and James Cunningham—AI security researchers at Carnegie Mellon University's Software Engineering Institute—discuss the growing threat of data poisoning in AI systems and highlight emerging mitigation strategies, including chain-of-custody controls.
In this episode of the SEI Podcast Series, Julie Lawler and James Cunningham—AI security researchers at Carnegie Mellon University's Software Engineering Institute—discuss the growing threat of data poisoning in AI systems and highlight emerging mitigation strategies, including chain-of-custody controls.
]]>While Stanford University found that AI investments, optimism, and accessibility are rising, a recent MIT report suggests that 95 percent of organizations are realizing no returns on their generative AI investments. Research from Accenture found that only 8 percent of companies are scaling AI at an enterprise level and embedding the technology into core business strategy to maximize value.
Mismatched expectations, misaligned applications, and poorly executed or untested implementation practices—not the technology itself—often keep organizations from realizing immediate value from an AI investment. For AI to increase efficiency, productivity, and value while conserving resources and lowering overall costs, organizations need to shift their focus from hype-driven experimentation to foundational capabilities and practical, measurable outcomes. In our latest podcast from the Carnegie Mellon University Software Engineering Institute, Dr. Ipek Ozkaya, technical director of AI-Native Software Engineering, sits down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss their work on an AI Adoption Maturity Model that organizations can use to create a roadmap for predictable AI adoption and realization of AI benefits.
Mismatched expectations, misaligned applications, and poorly executed or untested implementation practices—not the technology itself—often keep organizations from realizing immediate value from an AI investment. For AI to increase efficiency, productivity, and value while conserving resources and lowering overall costs, organizations need to shift their focus from hype-driven experimentation to foundational capabilities and practical, measurable outcomes. In our latest podcast from the Carnegie Mellon University Software Engineering Institute, Dr. Ipek Ozkaya, technical director of AI-Native Software Engineering, sits down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss their work on an AI Adoption Maturity Model that organizations can use to create a roadmap for predictable AI adoption and realization of AI benefits.
]]>
From early 2022 through late 2024, a group of threat actors publicly known as APT28 exploited known vulnerabilities, such as CVE-2022-38028, to remotely and wirelessly access sensitive information from a targeted company network. This attack did not require any hardware to be placed in the vicinity of the targeted company's network as the attackers were able to execute remotely from thousands of miles away. With the ubiquity of Wi-Fi, cellular networks, and Internet of Things (IoT) devices, the attack surface of communications-related vulnerabilities that can compromise data is extremely large and constantly expanding.
In the latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI) Joseph McIlvenny, a senior research scientist, and Michael Winter, vulnerability analysis technical manager, both with the SEI's CERT Division, discuss common radio frequency (RF) attacks and investigate how software and cybersecurity play key roles in preventing and mitigating these exploitations.
In the latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI) Joseph McIlvenny, a senior research scientist, and Michael Winter, vulnerability analysis technical manager, both with the SEI's CERT Division, discuss common radio frequency (RF) attacks and investigate how software and cybersecurity play key roles in preventing and mitigating these exploitations.
]]>Modern data analytic methods and tools—including artificial intelligence (AI) and machine learning (ML) classifiers—are revolutionizing prediction capabilities and automation through their capacity to analyze and classify data. To produce such results, these methods depend on correlations. However, an overreliance on correlations can lead to prediction bias and reduced confidence in AI outputs.
Drift in data and concept, evolving edge cases, and emerging phenomena can undermine the correlations that AI classifiers rely on. As the U.S. government increases its use of AI classifiers and predictors, these issues multiply (or use increase again). Subsequently, users may grow to distrust results. To address inaccurate erroneous correlations and predictions, we need new methods for ongoing testing and evaluation of AI and ML accuracy. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Nicholas Testa, a senior data scientist in the SEI's Software Solutions Division (SSD), and Crisanne Nolan, and Agile transformation engineer, also in SSD, sit down with Linda Parker Gates, Principal Investigator for this research and initiative lead for Software Acquisition Pathways at the SEI, to discuss the AI Robustness (AIR) tool, which allows users to gauge AI and ML classifier performance with data-based confidence.
Drift in data and concept, evolving edge cases, and emerging phenomena can undermine the correlations that AI classifiers rely on. As the U.S. government increases its use of AI classifiers and predictors, these issues multiply (or use increase again). Subsequently, users may grow to distrust results. To address inaccurate erroneous correlations and predictions, we need new methods for ongoing testing and evaluation of AI and ML accuracy. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Nicholas Testa, a senior data scientist in the SEI's Software Solutions Division (SSD), and Crisanne Nolan, and Agile transformation engineer, also in SSD, sit down with Linda Parker Gates, Principal Investigator for this research and initiative lead for Software Acquisition Pathways at the SEI, to discuss the AI Robustness (AIR) tool, which allows users to gauge AI and ML classifier performance with data-based confidence.
]]>How can you ever know whether an LLM is safe to use? Even self-hosted LLM systems are vulnerable to adversarial prompts left on the internet and waiting to be found by system search engines. These attacks and others exploit the complexity of even seemingly secure AI systems.
In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), David Schulker and Matthew Walsh, both senior data scientists in the SEI's CERT Division, sit down with Thomas Scanlon, lead of the CERT Data Science Technical Program, to discuss their work on System Theoretic Process Analysis, or STPA, a hazard-analysis technique uniquely suitable for dealing with AI complexity when assuring AI systems.
In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), David Schulker and Matthew Walsh, both senior data scientists in the SEI's CERT Division, sit down with Thomas Scanlon, lead of the CERT Data Science Technical Program, to discuss their work on System Theoretic Process Analysis, or STPA, a hazard-analysis technique uniquely suitable for dealing with AI complexity when assuring AI systems.
]]>]]>
]]>
A strong cyber defense is vital to public- and private-sector activities in the United States. In 2019, in response to an executive order to strengthen America's cybersecurity workforce, the Department of Homeland Security's Cybersecurity and Infrastructure Security Agency (CISA) partnered with the SEI to develop and run the President's Cup Cybersecurity Competition, a national cyber competition that identifies and rewards the best cybersecurity talent in the federal workforce. In six years, more than 8,000 people have taken part in the President's Cup. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Jarrett Booz, technical lead for the President's Cup, and John DiRicco, a training specialist in the SEI's CERT Division, sit down with Matthew Butkovic, the CERT technical director of cyber risk and resilience, to reflect on six years of hosting the cup, including challenges, lessons learned, the path forward, and publicly available resources.
]]>
]]>
]]>
]]>
]]>
]]>
]]>
]]>
In this SEI Podcast, Shannon Gallagher, AI engineering team lead, and Rachel Dzombak, special advisor to the director of the SEI's AI Division, discuss the findings and recommendations from the Mayflower Project and provides additional background information about LLMs and how they can be engineered for national security use cases.
]]>In this SEI Podcast, Shannon Gallagher, AI engineering team lead, and Rachel Dzombak, special advisor to the director of the SEI's AI Division, discuss the findings and recommendations from the Mayflower Project and provides additional background information about LLMs and how they can be engineered for national security use cases.
]]>]]>
]]>
As in commercial companies, DoD PMs are accountable for the overall cost, schedule, and performance of a program. The PM's job is even more complex in large programs with multiple software-development pipelines where cost, schedule, performance, and risk for the products of each pipeline must be considered when making decisions, as well as the interrelationships among products developed on different pipelines. Nichols and Cohen discuss how PMs can collect and transform unprocessed DevSecOps development data into useful program-management information that can guide decisions they must make during program execution. The ability to continuously monitor, analyze, and provide actionable data to the PM from tools in multiple interconnected pipelines of pipelines can help keep the overall program on track.
]]>As in commercial companies, DoD PMs are accountable for the overall cost, schedule, and performance of a program. The PM's job is even more complex in large programs with multiple software-development pipelines where cost, schedule, performance, and risk for the products of each pipeline must be considered when making decisions, as well as the interrelationships among products developed on different pipelines. Nichols and Cohen discuss how PMs can collect and transform unprocessed DevSecOps development data into useful program-management information that can guide decisions they must make during program execution. The ability to continuously monitor, analyze, and provide actionable data to the PM from tools in multiple interconnected pipelines of pipelines can help keep the overall program on track.
]]>Smith describes his experiences at NASA's Katherine Johnson IV&V Facility as a project manager for the Orion IV&V team. On that project, the developer employed Scaled Agile Framework (SAFe) as their development process, which had challenging consequences for established IV&V practices within NASA IV&V. Smith also discusses the ways in which NASA adapted to this change and describes strategies and tactics for reconciling Agile and IV&V.
]]>Smith describes his experiences at NASA's Katherine Johnson IV&V Facility as a project manager for the Orion IV&V team. On that project, the developer employed Scaled Agile Framework (SAFe) as their development process, which had challenging consequences for established IV&V practices within NASA IV&V. Smith also discusses the ways in which NASA adapted to this change and describes strategies and tactics for reconciling Agile and IV&V.
]]>In this podcast, Touhill discusses topics including the need for systems to be secure by design and secure by default, the importance of transparency in the reporting of vulnerabilities and anomalous system behavior, the CERT Acquisition Security Framework, the need to secure data across a wide range of disparate devices and systems, and tactics and strategies for individuals and organizations to safeguard their data and the systems they rely on daily.
]]>In this podcast, Touhill discusses topics including the need for systems to be secure by design and secure by default, the importance of transparency in the reporting of vulnerabilities and anomalous system behavior, the CERT Acquisition Security Framework, the need to secure data across a wide range of disparate devices and systems, and tactics and strategies for individuals and organizations to safeguard their data and the systems they rely on daily.
]]>]]>
]]>
In this podcast from the Carnegie Mellon University Software Engineering Institute, Shannon Gallagher, a data scientist with SEI's CERT Division, and Dominic Ross, multimedia team lead for the SEI, discuss deepfakes, their exponential growth in recent years, their increasing technical sophistication, and the problems they pose for individuals and organizations. Gallagher and Ross also discuss the SEI's recent research in assessing the technology underlying the creation and detection of deepfakes and understanding current and future threat levels.
]]>
]]>
]]>
]]>
]]>
]]>
]]>
The Future of Cyber Podcast Series explores whether we can use the innovations of the past to address the problems of the future. In this SEI Podcast, David Hickton, founding director of the University of Pittsburgh Institute for Cyber Law, Policy, and Security, sits down with Bobbie Stempfley, director of the SEI's CERT Division, to talk about the future of cybercrime.
]]>The Future of Cyber Podcast Series explores whether we can use the innovations of the past to address the problems of the future. In this SEI Podcast, David Hickton, founding director of the University of Pittsburgh Institute for Cyber Law, Policy, and Security, sits down with Bobbie Stempfley, director of the SEI's CERT Division, to talk about the future of cybercrime.
]]>As this podcast details, the report provides a snapshot of best practices and biggest challenges along with three guides for implementing cyber intelligence with artificial intelligence, the internet of things, and public cyber threat frameworks.
Lead author Jared Ettinger discusses the findings of the report, which the SEI conducted on behalf of the U.S. Office of the Director of National Intelligence.
]]>As this podcast details, the report provides a snapshot of best practices and biggest challenges along with three guides for implementing cyber intelligence with artificial intelligence, the internet of things, and public cyber threat frameworks.
Lead author Jared Ettinger discusses the findings of the report, which the SEI conducted on behalf of the U.S. Office of the Director of National Intelligence.
]]>"This is one case where the military or the government can learn from industry, sort of a spin-in to the government. The government has traditionally followed other approaches that were very requirements-based. They have perfected requirements engineering. What we have found is that in many cases with software systems, we really don't know the requirements when we start, not completely, and they evolve with time as users start to experience the software."
"This is one case where the military or the government can learn from industry, sort of a spin-in to the government. The government has traditionally followed other approaches that were very requirements-based. They have perfected requirements engineering. What we have found is that in many cases with software systems, we really don't know the requirements when we start, not completely, and they evolve with time as users start to experience the software."
]]>"If you discover [software defects] at system integration test, the cost of fixing a problem is 300 to 1,000 times higher than doing it upfront. So if upfront, you spent $10,000 fixing it, it's between $3 and $10 million on the backend that you are saving by the way."
"If you discover [software defects] at system integration test, the cost of fixing a problem is 300 to 1,000 times higher than doing it upfront. So if upfront, you spent $10,000 fixing it, it's between $3 and $10 million on the backend that you are saving by the way."
]]>"There is never enough time, money, power, resources—whatever it is—and we make design tradeoffs. Adversaries are looking at what opportunities that creates. They are looking at failures in implementation."
"There is never enough time, money, power, resources—whatever it is—and we make design tradeoffs. Adversaries are looking at what opportunities that creates. They are looking at failures in implementation."
]]>"If you look at large organizations like the DoD, they have embraced this. They are looking to buy infrastructures as a service and even moving office automation to the cloud. For smaller organizations, though, it is something of a challenge, so we wanted to look at and give people some ideas about the challenges they will face when they do this."
"If you look at large organizations like the DoD, they have embraced this. They are looking to buy infrastructures as a service and even moving office automation to the cloud. For smaller organizations, though, it is something of a challenge, so we wanted to look at and give people some ideas about the challenges they will face when they do this."
]]>"Part of it is the ability to use a wide variety of tools to answer questions about what is happening on the network and to figure out ways to go past inference and supposition and to get facts that can actually provide support for the hypothesis that you're coming up with.
"Part of it is the ability to use a wide variety of tools to answer questions about what is happening on the network and to figure out ways to go past inference and supposition and to get facts that can actually provide support for the hypothesis that you're coming up with.
]]>"A chronology naturally fell out that gave a temporal description of how a particular incident unfolded. So we can see precursor events that foreshadowed the event or the escalation of events that were to
"A chronology naturally fell out that gave a temporal description of how a particular incident unfolded. So we can see precursor events that foreshadowed the event or the escalation of events that were to
]]>