Biden-era AI safety promises aren’t holding up, and Apple’s the weakest link

Throughout 2023, the Biden administration persuaded a group of AI companies to sign voluntary commitments to “drive safe, secure, and trustworthy development of AI technology.” But the world of politics—and the AI industry—moves quickly.

A new analysis by researchers at Brown, Harvard, and Stanford suggests that only about half of those commitments are still being honored. The worst-performing signatory, Apple, has evidence for just one in eight. All commitments were, and remain, voluntary.

“This came at a time politically when we expected things might be changing,” says Rishi Bommasani, a Stanford researcher and one of the study’s authors. The analysis, posted as a preprint on arXiv, notes that the commitments were made late in the Biden administration, and the political climate has since shifted in Washington, D.C.

The researchers translated the White House’s eight voluntary AI commitments—covering safety testing, security, and public trust—into 30 yes-no indicators, then scored 16 signatory companies based on public disclosures up to December 31, 2024. “It’s difficult to discern what exactly constitutes a company satisfying their commitment adequately versus not,” says Bommasani.

With no clear guidance from the White House, they adopted a simple test: “Is there any evidence publicly that the companies are doing something in furtherance of these commitments?” While it’s possible a company could be complying without disclosing it, Bommasani argues the lack of transparency itself is a problem.

The biggest names in AI performed best. OpenAI scored highest at 83%, followed by Anthropic (80%), Google (77%), Microsoft (73%), and Amazon and Meta (both 67%). Seven companies overall were seen as upholding the commitments by the end of 2024. Apple ranked last, at just 13%. (The company did not respond to a request for comment.) Its lower score may partly reflect the fact it joined a year later than most others, in July 2024.

Across all 16 companies, the average score was 52%. “This is one of those things where, you have some general intuition of, probably companies are doing fairly well, and maybe we need to raise the bar,” says Bommasani. “But once you actually go and dive into the weeds, it becomes clear that actually the reality is quite different.”

That matters, he argues, because policymakers are weighing voluntary versus mandatory regulation. “Many jurisdictions are thinking about how do we balance more voluntary approaches versus more mandatory regulatory tools,” he says. The voluntary approach, he concludes, “don’t seem that useful.”