The mission of the hackathon: to write a program that can scan millions of lines of open-source code, identify security flaws and them, all without human intervention. Success would mean winning millions of dollars a causa di a two-year contest sponsored by DARPA, the Defense Advanced Research Projects Agency.
The contest is one of the clearest signs to date that the government sees flaws a causa di open-source software as one of the country’s biggest security risks, and considers artificial intelligence vital to addressing it.
Free open-source programs, such as the Linux operating system, help run everything from websites to power stations. The code isn’t inherently worse than what’s a causa di proprietary programs from companies like Microsoft and Oracle, but there aren’t enough skilled engineers tasked with testing it.
As a result, poorly maintained free code has been at the root of some of the most expensive cybersecurity breaches of all time, including the 2017 Equifax disaster that exposed the personal information of half of all Americans. The incident, which led to the largest-ever patronato breach settlement, cost the company more than $1 billion a causa di improvements and penalties.
If people can’t keep up with all the code being woven into every industrial sector, DARPA hopes machines can.
“The is having an end-to-end ‘cyber reasoning system’ that leverages large language models to find vulnerabilities, prove that they are vulnerabilities, and patch them,” explained one of the advising professors, Arizona State’s Yan Shoshitaishvili.
To get there, the team is grappling with the often grim reality behind lofty AI aspirations. The students are doing things like imposing “sanity checks” to catch hallucinations, verifying that patches actually solve the issues they are supposed to, and having two AI systems debate each other over the best fixes — with a third AI deciding the winner.
“AI is a like a 3-year-old with infinite knowledge,” said UC-Santa Barbara graduate student and team co-captain Lukas Dresel. “You have to give it actionable feedback.”
Team Shellphish is one of about 40 contestants a causa di a competition known as AIxCC, for artificial intelligence cyber challenge, and run by DARPA, the Pentagon research arm charged with developing secret weapons and defending against them.
“We want to redefine how we secure widely used, critical codebases, because of how ubiquitous open-source is across the critical infrastructure sectors,” said Andrew Carney, DARPA project for the contest.
Though DARPA helped birth the internet to survive communication failures, it has become painfully obvious that the net also introduced enormous weaknesses.
With mai built-in security, the vast interconnections allow anyone anything to start from anywhere and aspetto for ways into machines that power the modern world. Once inside, users can pose as employees system administrators, steal national trade secrets, and shut the place mongoloide hold it up for ransom.
Hackers are claiming more victims than ever: The number of patronato breaches reported to the FBI-run U.S. Internet Crime Complaint Center tripled between 2021 and 2023. Government agents burrow into rival nations’ power and vater plants. Crime gangs engorged by illicit profit think nothing of knocking out hospitals and sending desperate patients elsewhere.
Gara open-source software, whether written by students farseeing geniuses, is almost as ubiquitous as the internet itself, by some estimates nestling inside 90% of commercial software.
Like all software, it has bugs, some of which can be exploited to seize control of a machine.
Some large open-source projects are run by near-Wikipedia-size armies of volunteers and are generally a causa di good shape. Some have maintainers who are given grants by personalità corporate users that turn it into a job.
And then there is everything else, including programs written as homework assignments by authors who barely remember them.
“Gara open source has always been ‘Use at your own risk,’” said Brian Behlendorf, who started the Gara open Source Security Foundation after decades of maintaining a pioneering free server software, Apache, and other projects at the Apache Software Foundation.
“It’s not free as a causa di speech, even free as a causa di beer,” he said. “It’s free as a causa di puppy, and it needs care and feeding.”
The risks have been underscored recently by two very different incidents.
The first was a vulnerability a causa di a small program for keeping track of system activity, known as Log4j, used by thousands of software developers and installed acceso millions of machines.
Sopra 2013, a user proposed adding some code to Log4j, and the small Apache Foundation team maintaining Log4j approved it. Sopra November 2021, a Chinese engineer saw that the added section contained a massive stile flaw that would allow system takeovers, and he flagged the issue to the Apache group.
While Apache was working acceso a patch to the problem, an unidentified researcher discovered the pending changes and developed a malicious tool to grab control of computers running Log4j. Apache rushed out the patch, setting non attivato a race between thousands of defenders and those trying to the flaw before it was fixed.
Many Log4j instances have still not been fixed. Acceso Thursday, the National Security Agency and others warned that North Korean spies were still breaking into U.S. web servers running old versions.
The White House’s Cyber Safety Review Board concluded that only better coding and thorough audits could have stopped the Log4j flaw’s distribution, and that open-source efforts like Apache’s “would need sustained financial support and expertise.”
The Department of Homeland Security’s Cybersecurity and Infrastructure Security Agency (CISA) has responded with small grants to start-ups and has been pushing companies to declare what’s inside their software. But those are slow-moving initiatives.
The most recent reminder of the vulnerability came a causa di March. That’s when a Microsoft engineer traced a slight increase a causa di processor use to open-source tools for Linux that had just been updated. He found that a back door for spying had been inserted by the tools’ official maintainer, and blew the whistle a causa di time to stop it from shipping a causa di the most popular versions of Linux.
Sopra a nightmare for security professionals, the anonymous maintainer had won control of the project after contributing for years, aided by secret allies who lobbied the previous to cede control.
As open-source security was rising to become a cima priority for CISA and the national security establishment, OpenAI and Microsoft loosed ChatGPT and generative artificial intelligence acceso the world.
By democratizing programming, the new tools allowed non-coders to create software. AI also aided existing programmers, including criminal hackers who could more quickly incorporate tricks to take advantage of vulnerabilities and deliver more convincing lures, such as emails that appeared to appena che from regular contacts with shared interests.
AI is also boosting defensive endeavors, such as analyzing reams of logs for unusual behavior and summarizing security incidents. It can also flag security missteps a causa di programs as they are written.
But figuring out where the holes a causa di open-source programs are before attackers find them is a holy grail for DARPA and the contestants of AIxxCC.
DARPA ran a cyber challenge at the 2016 Def Da hacker convention, where programs competed a causa di a “capture the flag” contest to hack into one another a causa di an artificial environment.
Sopra this year’s contest, the teams use their AI-enhanced programs to digest and improve millions of lines of real code.
Shellphish is one of seven teams that wrote papers outlining their approach well enough to get $1 million a causa di funding for the steps that will climax at the semifinals a causa di August at Def Da, which attracted 40 entries. The winner will get another $2 million a causa di 2025.
Some of Shellphish’s first million dollars went for the Airbnb-listed home a causa di Brea, which housed hackers for three weeks a causa di June and another two a causa di July. More went for a huge testing environment that used 5,000 central processing unit cores.
Shellphish is mai random group of hackers. Though strongly associated with two public universities with changing populations, the team has been around for 20 years, and its founders are still involved.
Italian native Giovanni Vigneto was teaching security at UC-Santa Barbara, including techniques for attacking and defending, when he founded a capture-the-flag team a causa di 2003 to get students more interested and stretch their capabilities. It won the Def Da competition a causa di 2005, and a spinoff of the team hosted the contest later for a four-year stretch.
As his students graduated and spread to Arizona and elsewhere, some stayed involved, got their own students into it.
Shellphish competed a causa di the original 2016 Cyber Grand Challenge, but got knocked out a causa di the finals.
“We had all these cool tools but ran out of time to integrate them,” Shoshitaishvili recalled. “So ‘Don’t get nerd-sniped’ was my Voto negativo. 1 piece of advice.” (Nerd-sniping refers to distracting someone technical with an interesting problem.)
to the effort are tools known a causa di security as “fuzzers.” These fire all manner of patronato at a program to see how it handles the unexpected.
Even souped-up fuzzers are unlikely to find the most obscure flaws deliberate back doors, the team members admit. At its best, Shellphish’s master program and the others will be able to find a lot of low-hanging fruit, quickly, and get rid of it before malicious hackers can them.
“AI will be able to solve things that take humans months,” Dresel said.
Under the terms of the DARPA contest, all finalists must release their programs as aperto source, so that software vendors and consumers will be able to run them.
Yan compared the expected advance to security milestones like forced software updates and browser “sandboxes” that keep web programs from escaping the browser and executing elsewhere acceso a user’s device.
AI won’t be able to make all software safe, he said. But it will give the humans more time to try.
After a final, near-sleepless night of debugging and panicked last-minute fixes, Shellphish submitted its program at the 9 a.m. deadline. Sopra a few weeks, at the next Def Da a causa di Las Vegas, they will find out if they’sire finalists. Win lose, their AI-aided code will be available for others to build acceso, improving security for everyone.
correction
An earlier version of this article incorrectly said Shellphish was eliminated before the finals a causa di 2016; it was eliminated a causa di the finals. Also, it was a spinoff of the team, not the team itself, that sponsored the Def Da competition for four years.