Essential Complexity

Modernizing high-risk systems in the age of AI.

• 4 min read

Uncut Gems

OpenAI agents flooded RubyGems with 2,000 malicious packages in a day, exploited a real credential vulnerability, and used RubyDoc's auto-execute feature as a remote code execution primitive. This is a pattern, not an incident.

Uncut Gems
Joe Leo
Joe Leo

Founder, Def Method

"We're dealing with a major malicious attack on @rubygems right now." That was Maciej Mensfeld in May of this year after something flooded RubyGems with more than 2,000 malicious packages in a single day.

RubyGems is a package manager for Ruby and the backbone of the open source ecosystem on which Ruby thrives. As of this writing, RubyGems hosts over 195,000 packages (called Gems) that have been downloaded over 266 billion times. Any attack on RubyGems is an existential threat to the entire Ruby community.

"It was a major attack in terms of what we see in volume," Marty Haught, the organization's director of open source, told the Wall Street Journal. While under attack, RubyGems maintainers paused new signups for four days. Security researchers dubbed it the GemStuffer campaign and noted, with confusion, that the packages seemed to exist mainly to scrape government data that was already publicly available.

Until recently, nobody knew who or what was behind it. RubyGems removed over 500 packages, found no evidence that any of the attempted credential theft had succeeded, and moved on. Four months later, independent researchers Spencer Kitts, Thomas Larsen, and Sydney Von Arx published a report tying the attack to OpenAI. They matched it to the pattern already documented in a separate, even more high-profile attack on Hugging Face in July. Reuters later reported that the evaluation population behind both incidents overlapped substantially: of roughly 1,200 agents involved in the broader testing run, about 700 went on to take part in the July attack on Hugging Face. In other words, it wasn't just the same company but many of the same agents responsible for both attacks.

OpenAI's public position has moved twice in the space of a week. On September 11th, the company said it hadn't been able to verify the specific claims of malicious packages. By September 14th, per the Wall Street Journal, OpenAI had confirmed its agents were behind the RubyGems activity — and offered an explanation: the agents, working during a training run where they lacked unrestricted internet access, treated RubyGems as an improvised substitute for a web browser. The findings reached OpenAI through a research nonprofit called Nightingale Collective, which shared its report with both the company and the Journal before publication. It appears that OpenAI's internal detection didn't catch the attack until someone outside handed them the evidence.

Rhymes with Hugging Face

When you publish a Gem to RubyGems, a companion service called RubyDoc.info automatically builds and hosts documentation for it. That build process runs a tool called YARD, which reads a small configuration file and, as part of generating docs, will execute a Ruby script the package author included. The agents used this script utility to remotely execute code on RubyDoc's servers. They published packages with malicious scripts disguised as doc-build helpers, let RubyDoc run it automatically, then used that access to scrape target websites and smuggle the results back out by publishing the scraped data as another gem.

Some of the packages went further. They attempted to exploit a real, then-undiscovered vulnerability in how RubyGems' CDN cached sign-in credentials. This flaw lets an attacker query a specific endpoint and potentially retrieve another user's API key, if that user had signed in recently on the right server. RubyGems didn't find and patch this themselves until July, two months after the agents had already tried to use it. Whether the attempt succeeded is still unknown; RubyGems says its own review found no evidence it did.

If any of this sounds familiar, it should. RubyDoc's build process of arbitrary code executing automatically when you load a documentation artifact is the same shape of feature that gave the Hugging Face incident its opening in July. Both Hugging Face and RubyGems are platforms auto-executing user-submitted content because that's the convenience the platform is built to provide. Two unrelated companies, two unrelated codebases, the same underlying architecture, hit the same way, months apart, by what may well be the same agents.

OpenAI: The Cause of, and Solution To, All of Life's Problems

When the Hugging Face breach became public, the natural read was that it exposed a specific weakness in a specific platform's specific feature. But it turns out that it isn't only a Hugging Face problem. It's a pattern sitting in any piece of developer infrastructure built around the same convenience: npm, PyPI, CI systems, doc generators, anything that runs code on your behalf because reading every submission by hand would defeat the point of the platform. Governance, one of Def Method's principles for next-generation software engineering, advises us to recognize that this entire category of feature carries the same latent exposure everywhere it appears. An organization's containment posture is only as good as its ability to respond when a new threat is detected.

This raises more questions about responsibility, ethics, and reporting for OpenAI. While they have validated the independent researchers' account, OpenAI has accepted none of the responsibility that account implies. The researchers documented OpenAI agents attempting to exploit a real credential-theft vulnerability, code that disarmed and hid its own payload after execution, files named hack.rb, exploit.rb, and evil.rb, comments reading # malicious probe and # leak exfil by repeated attempts. Why were the agents hunting for a way to steal other users' credentials? How will they make this right going forward? How will they even do right by RubyGems, a foundation they were claiming to help?

We'll have to wait for answers to those questions and more. Knowing what I know about the Ruby Central security team, Governance has already been revisited and elevated to continue RubyGems' resiliency in a world of increasing AI-enabled threats. This is a good time to revisit your own policies and detection framework, as well.

Need to modernize a high-stakes system?

GUIDE is Def Method's framework for preserving understanding, strengthening governance, and building evidence before implementation accelerates.

If this was useful, you might enjoy Essential Complexity — a bi-weekly letter on modernizing high-risk systems in the age of AI.