A few federal agencies have started to use sophisticated artificial intelligence tools to help deal with immense caseloads of Freedom of Information Act requests, but some transparency advocates warn that the government needs additional safeguards before more widely deploying the technology.
At least three agencies — the State Department, the Justice Department and the Centers for Disease Control and Prevention — have tried out or are now testing machine-learning models and algorithms to help search for information in repositories holding billions of government records, federal officials confirmed to NBC News in recent interviews.
Officials from multiple agencies also have separately tested an AI prototype called “FOIA Assistant” that’s being developed by a federally funded research group as a possible model for dealing with record-high numbers of new requests and growing backlogs of existing ones.
“There is no way for FOIA to work in the future unless you can automate searching of the millions, hundreds of millions, billions of records that these government agencies hold,” said Jason R. Baron, a University of Maryland information studies professor and leading expert on the use of artificial intelligence in government access. “The problem is simply unsolvable without AI.”
Unlike the American legal system, which for years has used court-approved “eDiscovery” technologies to help find and extract sensitive information from documents exchanged during litigation, the use of artificial intelligence for FOIA purposes is in its infancy, he said.
Still, some open government and civil rights advocates are already raising concerns that the government’s move toward using AI to help address FOIA problems may create new ones.
Adam Marshall, a senior staff attorney for the nonprofit government watchdog Reporters Committee for Freedom of the Press, said he has high hopes that AI and other technology will help make more information available to the public faster. But first, he said, it’s necessary to understand how the technology “is being trained and used by humans.”
So far, government agencies haven’t widely disclosed to the public what kinds of AI tools are being used, and in what fashion, Marshall said. He added he worries that overburdened FOIA officers introduced to AI may become too reliant on or complacent with machines to make decisions that typically require thoughtful legal analysis.
“There need to be clear standards for the use of this technology and assurances that they’re being followed,” Marshall said. “There also need to be procedures in place for challenging decisions where machine algorithms are used, including when they could be unnecessarily or illegally withholding information.”
Signed into law in 1967, the Freedom of Information Act is meant to ensure government transparency and access to information by requiring agencies to provide records to citizens who make requests.
But experts widely agree the FOIA process must be modernized and fixed, as requests can sometimes take months, even years, to fulfill. An increasing number of requesters have turned to the courts for help in prying records loose in a timely manner.
Last year, the 120 federal agencies subject to the federal disclosure law collectively received more than 928,000 FOIA requests — an all-time high and 90,000 more than in 2021, according to the Justice Department’s Office of Information Policy. Meanwhile, the number of backlogged requests in 2022 — nearly 207,000 — also reached record territory, up nearly 50,000 from the previous year.
The backlogs are ballooning at a time when agencies are anticipating a boom in storage and disclosure of electronic records, several officials said.
In turn, a group of FOIA officers have been spreading the word on “the awareness and availability of these types of tools” as part of a push to improve the FOIA process, said Michael Sarich, who oversees FOIA issues for the Department of Veterans Affairs and co-chairs the Chief FOIA Officers Council‘s technology subcommittee tasked with exploring the use of AI.
“The volume of information and records and data that’s now available was unimaginable 56 years ago when the FOIA was created,” added Eric F. Stein, another subcommittee co-chair and deputy assistant secretary for the State Department who oversaw FOIA initiatives, declassification strategies and other record management planning for the agency.
The state department is now testing two AI models to help process FOIA requests, Stein said. One model employs machine-learning algorithms to find records in the agency’s centralized databases and archives, which hold more than 3 billion records, he said.
The other pilot sends prompts to people submitting FOIA requests via the agency’s web portal based on the words they input, suggesting where they might be able to find information that’s already publicly available or that they narrow the scope of their requests to help facilitate quicker responses, he said.
A Justice Department spokesperson said the use of AI with FOIA processing is “a purely exploratory” endeavor at this point and not a policy at the agency.
The CDC also recently “explored using an AI software tool to analyze documents uploaded into the FOIA system,” a spokesperson told NBC News. “Ultimately, this software did not meet the needs of the office in an efficient and timely manner and it was determined the tool was not the right fit (for the) intended use.”
Some federal agencies have started testing another prototype, called “FOIA Assistant,” that helps locate records within vast government datasets and suggests redactions of information under at least three of the law’s nine categories of exemptions.
“There’s really no tool like this that helps FOIA analysts out there commercially that we’re aware of,” said Bradford Brown, the project’s outcome lead for the Mitre Corp., a nonprofit manager of federally funded government research and development projects that built the prototype.
To help train and test the model, Brown said Mitre developers partnered with FOIA analysts to curate datasets by identifying and annotating portions of records exempt from disclosure because they contain “deliberative language” intended to help officials make decisions.
Baron, former litigation director at the National Archives and Records Administration, said he helped test the prototype by annotating hundreds of Clinton administration policy documents, and found an early version of the technology to be about 70 percent accurate.
“It’s not perfect,” Baron said. “But using this type of AI actually could be of enormous help in the future when agencies routinely are finding tens or hundreds of thousands of potentially responsive records that they otherwise would have to review manually, a process that almost assuredly will take many years.”
Brett Max Kaufman, a senior staff attorney for the ACLU who specializes in surveillance and national security issues, agreed that artificial intelligence models may ultimately hasten the release of government information, but he cautioned that benefit could come at a cost.
“Agencies regularly over-redact and over-withhold information under FOIA,” he said. “There’s a culture of keeping things as secret as possible for as long as possible, in part because all of the incentives run in that direction. And if you’re just teaching a machine how to do the same thing that you’ve always done, it has the potential to make things even worse.”
Brown, who noted the Mitre prototype is still a work in progress, declined to say which of “multiple agencies” have so far tested it. He added that the tool is merely meant to be a “cognitive assistant to help analysts.”
“In the end, humans are still going to have to make the decisions,” he said
